[Openais] Re: [Saftest-devel] [ANNOUNCE] saftest_AIS-B.01.01_0.1.0 released!

Steven Dake sdake at mvista.com
Thu Feb 3 13:00:29 PST 2005


Crystal

Thanks for the early work on a B implementation of the test suite.  As
you may know openais is quickly moving to the B implementation in the
next few weeks and an early test suite will really help us out.  If you
need order of priority, we are doing conversions something like
CLM (done)
EVT (done)
CKPT (to be done)
AMF (may be next release)

I have a few comments on this test package.

I fired up the clm tests against our first ported service (the cluster
membership API B.01.01).  The test suite runs well after I figured out a
few things.

First of all, PLEASE integrate the timeout code into the run_tests.sh
script.  It took me a few hours to determine there was indeed a need for
an external script which I didn't have and had a hard time finding (but
eventualy found).  Then after I found the script, the calling parameters
didn't match the script I have so I had to figure out how to hack
run_tests.sh and the timeout script together.  External dependencies in
test suites on scripts with changing interfaces is a bad bad idea.

I also found a serious defect:
it appears test cases fail for the test cases that use the exec call. 
An example of this is saClmDispatch/8-1.c

int main(int argc, char *argv[])
{
        char same_case[]="./1-2.test";

        execl(same_case, NULL);
        printf("        Can't find case %s\n", same_case);
        return SAF_TEST_UNKNOWN;
}

Unfortunately we are not in the correct directory but instead in the top
level directory, which makes these test cases fail (execl can't find the
application).  This results in about 62 "Unknown reason" failures which
is over half the test cases.

In a perfect world the tests should be runnable from any directory
either top level or in the directory themselves.  This would be the
ideal scenario for developers (so individual test cases can be
debugged).

I really dislike having to say pass fail for the manual tests in
report.sh.  This is terribly annoying and skews the results of my test
cases because I either always answer pass or fail.  A command line
option to report or not manual tests would solve the problem.  Another
option should be added as well, which is "NOT TESTED".

Once saftest has these few issues fixed up, I'll post some results on
the B-01-01 implementation of CLM that has been developed in the openais
project.

Thanks for the hard work

Regards
-steve

On Thu, 2005-02-03 at 03:13, Xiong, Crystal wrote:
> The first release of AIS B.01.01 tests is available at http://sourceforge.net/projects/saftest. This release includes the tests for AIS-B.01.01 membership service.
> 
> >From this release, we will separate the releases for HPI, AIS and A.01.01, B.01.01. The new release is named as saftest_<SPEC-NAME>_<VERSION>, this release is named as saftest_AIS-B.01.01_0.1.0
> The existed releases for SAF A.01.01 will remain the same. Bug fix version for A.01.01 will be named as saftest_<VERSION>, such as saftest_1.0.1.
> 
> The major changes in this release are:
>      - Create AIS B.01.01 clm test cases (Thanks YuLing and YuPing)
>      - Support manual cases in test framework. (Thanks YuLing and XiaoWei)
>      - Update test framework scripts to support different releases we will make. (Thanks YuLing).
>      - Create scripts under utilities to facilate our work (Thanks XiaoWei)
>      - Update test framework document to v0.9 (Thanks YuLing)
> 
> Test case contribution and test execution are welcome!
> 
> The README in the release package describes how to use the framework, how to contribute more tests to the framework in a brief. For more details, please refer to Framework_Specification.htm document under doc directory in the release package.
> 
> The goal of this project is to create conformance test suite for SAF spec, including AIS A.01.01, AIS B.01.01, HPI A.01.01 and HPI B.01.01 spec. 
> 
> Any more comments, please discuss in the following mailing list:
> saftest-devel at lists.sourceforge.net (For the whole project) saftest-hpi-devel at lists.sourceforge.net (For HPI part) saftest-ais-devel at lists.sourceforge.net (For AIS part)
> 
> The webpage of this project can be found here: http://saftest.sourceforge.net/
> 
> Thanks,
> Crystal    
> 
> * This is my personal opinion *
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
> Tool for open source databases. Create drag-&-drop reports. Save time
> by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> _______________________________________________
> Saftest-devel mailing list
> Saftest-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/saftest-devel




More information about the Openais mailing list