[Openais] RE: [Saftest-hpi-devel] RE: [Saftest-ais-devel] Re: [Saftest-devel] [ANNOUNCE] saftest_AIS-B.01.01_0.1.0 released!

Steven Dake sdake at mvista.com
Fri Feb 4 11:15:10 PST 2005


This sounds good Ling

Regards,
-steve

On Thu, 2005-02-03 at 20:14, Yu, Ling L wrote:
> Steve,
> A more detail ideas about manual test cases come to my mind, 
> 1. Report.sh with no option, all automatica test cases will be
> calculated
> 2. Report.sh with -m option
>            List all manual test cases to user firstly, and ask user
> "Please make sure all manual cases have been run, enter Yes/No?", If
> user enter "Yes", the command will let user enter the manual cases
> results, like "Enter  **/**/manual/2.test result Pass/Fail/Untested?".
> The results will be calculated together with all automatic test cases. 
> 3. Add a script into saftest/utilities, to help to list all test cases
> according different options, such as "-m" for manual case, "-a" for
> automatic test cases
> 
> Do you any suggestion? 
> 
> Thanks
> - Ling
> 
> 
> -----Original Message-----
> From: saftest-hpi-devel-admin at lists.sourceforge.net
> [mailto:saftest-hpi-devel-admin at lists.sourceforge.net] On Behalf Of Yu,
> Ling L
> Sent: Friday, February 04, 2005 10:39 AM
> To: sdake at mvista.com; Xiong, Crystal; openais at lists.osdl.org
> Cc: saftest-devel at lists.sourceforge.net;
> saftest-hpi-devel at lists.sourceforge.net;
> saftest-ais-devel at lists.sourceforge.net
> Subject: [Saftest-hpi-devel] RE: [Saftest-ais-devel] Re: [Saftest-devel]
> [ANNOUNCE] saftest_AIS-B.01.01_0.1.0 released!
> 
> Steve
> Thanks for your good comments; it's very helpful for us. Pls see my
> comments as below.
> 
> Thanks
> - Ling
> 
> -----Original Message-----
> From: saftest-ais-devel-admin at lists.sourceforge.net
> [mailto:saftest-ais-devel-admin at lists.sourceforge.net] On Behalf Of
> Steven Dake
> Sent: Friday, February 04, 2005 5:00 AM
> To: Xiong, Crystal; openais at lists.osdl.org
> Cc: saftest-devel at lists.sourceforge.net;
> saftest-hpi-devel at lists.sourceforge.net;
> saftest-ais-devel at lists.sourceforge.net
> Subject: [Saftest-ais-devel] Re: [Saftest-devel] [ANNOUNCE]
> saftest_AIS-B.01.01_0.1.0 released!
> 
> Crystal
> 
> Thanks for the early work on a B implementation of the test suite.  As
> you may know openais is quickly moving to the B implementation in the
> next few weeks and an early test suite will really help us out.  If you
> need order of priority, we are doing conversions something like
> CLM (done)
> EVT (done)
> CKPT (to be done)
> AMF (may be next release)
> 
> I have a few comments on this test package.
> 
> I fired up the clm tests against our first ported service (the cluster
> membership API B.01.01).  The test suite runs well after I figured out a
> few things.
> 
> First of all, PLEASE integrate the timeout code into the run_tests.sh
> script.  It took me a few hours to determine there was indeed a need for
> an external script which I didn't have and had a hard time finding (but
> eventualy found).  Then after I found the script, the calling parameters
> didn't match the script I have so I had to figure out how to hack
> run_tests.sh and the timeout script together.  External dependencies in
> test suites on scripts with changing interfaces is a bad bad idea.
> [Yu, Ling L]  Yes, "timeout" are from netatalk-1.6.4-2.1, Thank for
> reminder. We will try to find a resolution about this issue. 
> I also found a serious defect:
> it appears test cases fail for the test cases that use the exec call. 
> An example of this is saClmDispatch/8-1.c
> 
> int main(int argc, char *argv[])
> {
>         char same_case[]="./1-2.test";
> 
>         execl(same_case, NULL);
>         printf("        Can't find case %s\n", same_case);
>         return SAF_TEST_UNKNOWN;
> }
> 
> Unfortunately we are not in the correct directory but instead in the top
> level directory, which makes these test cases fail (execl can't find the
> application).  This results in about 62 "Unknown reason" failures which
> is over half the test cases.
> 
> In a perfect world the tests should be runnable from any directory
> either top level or in the directory themselves.  This would be the
> ideal scenario for developers (so individual test cases can be
> debugged).
> [Yu, Ling L]  Yes, this is a defect; we can refine the run_tests.sh to
> fix it.
> 
> I really dislike having to say pass fail for the manual tests in
> report.sh.  This is terribly annoying and skews the results of my test
> cases because I either always answer pass or fail.  A command line
> option to report or not manual tests would solve the problem.  Another
> option should be added as well, which is "NOT TESTED".
> [Yu, Ling L]  I think add a command option is a good resolution. If
> "report.sh -m", all manual test cases will be calculated together. By
> default, only automatic test cases are considered. 
> Once saftest has these few issues fixed up, I'll post some results on
> the B-01-01 implementation of CLM that has been developed in the openais
> project.
> 
> Thanks for the hard work
> 
> Regards
> -steve
> 
> On Thu, 2005-02-03 at 03:13, Xiong, Crystal wrote:
> > The first release of AIS B.01.01 tests is available at
> http://sourceforge.net/projects/saftest. This release includes the tests
> for AIS-B.01.01 membership service.
> > 
> > >From this release, we will separate the releases for HPI, AIS and
> A.01.01, B.01.01. The new release is named as
> saftest_<SPEC-NAME>_<VERSION>, this release is named as
> saftest_AIS-B.01.01_0.1.0
> > The existed releases for SAF A.01.01 will remain the same. Bug fix
> version for A.01.01 will be named as saftest_<VERSION>, such as
> saftest_1.0.1.
> > 
> > The major changes in this release are:
> >      - Create AIS B.01.01 clm test cases (Thanks YuLing and YuPing)
> >      - Support manual cases in test framework. (Thanks YuLing and
> XiaoWei)
> >      - Update test framework scripts to support different releases we
> will make. (Thanks YuLing).
> >      - Create scripts under utilities to facilate our work (Thanks
> XiaoWei)
> >      - Update test framework document to v0.9 (Thanks YuLing)
> > 
> > Test case contribution and test execution are welcome!
> > 
> > The README in the release package describes how to use the framework,
> how to contribute more tests to the framework in a brief. For more
> details, please refer to Framework_Specification.htm document under doc
> directory in the release package.
> > 
> > The goal of this project is to create conformance test suite for SAF
> spec, including AIS A.01.01, AIS B.01.01, HPI A.01.01 and HPI B.01.01
> spec. 
> > 
> > Any more comments, please discuss in the following mailing list:
> > saftest-devel at lists.sourceforge.net (For the whole project)
> saftest-hpi-devel at lists.sourceforge.net (For HPI part)
> saftest-ais-devel at lists.sourceforge.net (For AIS part)
> > 
> > The webpage of this project can be found here:
> http://saftest.sourceforge.net/
> > 
> > Thanks,
> > Crystal    
> > 
> > * This is my personal opinion *
> > 
> > 
> > 
> > -------------------------------------------------------
> > This SF.Net email is sponsored by: IntelliVIEW -- Interactive
> Reporting
> > Tool for open source databases. Create drag-&-drop reports. Save time
> > by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> > Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> > _______________________________________________
> > Saftest-devel mailing list
> > Saftest-devel at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/saftest-devel
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
> Tool for open source databases. Create drag-&-drop reports. Save time
> by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> _______________________________________________
> Saftest-ais-devel mailing list
> Saftest-ais-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/saftest-ais-devel
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
> Tool for open source databases. Create drag-&-drop reports. Save time
> by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> _______________________________________________
> Saftest-hpi-devel mailing list
> Saftest-hpi-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/saftest-hpi-devel




More information about the Openais mailing list