[Openais] RE: [Saftest-hpi-devel] RE: [Saftest-ais-devel] Re:
[Saftest-devel] [ANNOUNCE] saftest_AIS-B.01.01_0.1.0 released!
Steven Dake
sdake at mvista.com
Fri Feb 4 11:15:10 PST 2005
This sounds good Ling
Regards,
-steve
On Thu, 2005-02-03 at 20:14, Yu, Ling L wrote:
> Steve,
> A more detail ideas about manual test cases come to my mind,
> 1. Report.sh with no option, all automatica test cases will be
> calculated
> 2. Report.sh with -m option
> List all manual test cases to user firstly, and ask user
> "Please make sure all manual cases have been run, enter Yes/No?", If
> user enter "Yes", the command will let user enter the manual cases
> results, like "Enter **/**/manual/2.test result Pass/Fail/Untested?".
> The results will be calculated together with all automatic test cases.
> 3. Add a script into saftest/utilities, to help to list all test cases
> according different options, such as "-m" for manual case, "-a" for
> automatic test cases
>
> Do you any suggestion?
>
> Thanks
> - Ling
>
>
> -----Original Message-----
> From: saftest-hpi-devel-admin at lists.sourceforge.net
> [mailto:saftest-hpi-devel-admin at lists.sourceforge.net] On Behalf Of Yu,
> Ling L
> Sent: Friday, February 04, 2005 10:39 AM
> To: sdake at mvista.com; Xiong, Crystal; openais at lists.osdl.org
> Cc: saftest-devel at lists.sourceforge.net;
> saftest-hpi-devel at lists.sourceforge.net;
> saftest-ais-devel at lists.sourceforge.net
> Subject: [Saftest-hpi-devel] RE: [Saftest-ais-devel] Re: [Saftest-devel]
> [ANNOUNCE] saftest_AIS-B.01.01_0.1.0 released!
>
> Steve
> Thanks for your good comments; it's very helpful for us. Pls see my
> comments as below.
>
> Thanks
> - Ling
>
> -----Original Message-----
> From: saftest-ais-devel-admin at lists.sourceforge.net
> [mailto:saftest-ais-devel-admin at lists.sourceforge.net] On Behalf Of
> Steven Dake
> Sent: Friday, February 04, 2005 5:00 AM
> To: Xiong, Crystal; openais at lists.osdl.org
> Cc: saftest-devel at lists.sourceforge.net;
> saftest-hpi-devel at lists.sourceforge.net;
> saftest-ais-devel at lists.sourceforge.net
> Subject: [Saftest-ais-devel] Re: [Saftest-devel] [ANNOUNCE]
> saftest_AIS-B.01.01_0.1.0 released!
>
> Crystal
>
> Thanks for the early work on a B implementation of the test suite. As
> you may know openais is quickly moving to the B implementation in the
> next few weeks and an early test suite will really help us out. If you
> need order of priority, we are doing conversions something like
> CLM (done)
> EVT (done)
> CKPT (to be done)
> AMF (may be next release)
>
> I have a few comments on this test package.
>
> I fired up the clm tests against our first ported service (the cluster
> membership API B.01.01). The test suite runs well after I figured out a
> few things.
>
> First of all, PLEASE integrate the timeout code into the run_tests.sh
> script. It took me a few hours to determine there was indeed a need for
> an external script which I didn't have and had a hard time finding (but
> eventualy found). Then after I found the script, the calling parameters
> didn't match the script I have so I had to figure out how to hack
> run_tests.sh and the timeout script together. External dependencies in
> test suites on scripts with changing interfaces is a bad bad idea.
> [Yu, Ling L] Yes, "timeout" are from netatalk-1.6.4-2.1, Thank for
> reminder. We will try to find a resolution about this issue.
> I also found a serious defect:
> it appears test cases fail for the test cases that use the exec call.
> An example of this is saClmDispatch/8-1.c
>
> int main(int argc, char *argv[])
> {
> char same_case[]="./1-2.test";
>
> execl(same_case, NULL);
> printf(" Can't find case %s\n", same_case);
> return SAF_TEST_UNKNOWN;
> }
>
> Unfortunately we are not in the correct directory but instead in the top
> level directory, which makes these test cases fail (execl can't find the
> application). This results in about 62 "Unknown reason" failures which
> is over half the test cases.
>
> In a perfect world the tests should be runnable from any directory
> either top level or in the directory themselves. This would be the
> ideal scenario for developers (so individual test cases can be
> debugged).
> [Yu, Ling L] Yes, this is a defect; we can refine the run_tests.sh to
> fix it.
>
> I really dislike having to say pass fail for the manual tests in
> report.sh. This is terribly annoying and skews the results of my test
> cases because I either always answer pass or fail. A command line
> option to report or not manual tests would solve the problem. Another
> option should be added as well, which is "NOT TESTED".
> [Yu, Ling L] I think add a command option is a good resolution. If
> "report.sh -m", all manual test cases will be calculated together. By
> default, only automatic test cases are considered.
> Once saftest has these few issues fixed up, I'll post some results on
> the B-01-01 implementation of CLM that has been developed in the openais
> project.
>
> Thanks for the hard work
>
> Regards
> -steve
>
> On Thu, 2005-02-03 at 03:13, Xiong, Crystal wrote:
> > The first release of AIS B.01.01 tests is available at
> http://sourceforge.net/projects/saftest. This release includes the tests
> for AIS-B.01.01 membership service.
> >
> > >From this release, we will separate the releases for HPI, AIS and
> A.01.01, B.01.01. The new release is named as
> saftest_<SPEC-NAME>_<VERSION>, this release is named as
> saftest_AIS-B.01.01_0.1.0
> > The existed releases for SAF A.01.01 will remain the same. Bug fix
> version for A.01.01 will be named as saftest_<VERSION>, such as
> saftest_1.0.1.
> >
> > The major changes in this release are:
> > - Create AIS B.01.01 clm test cases (Thanks YuLing and YuPing)
> > - Support manual cases in test framework. (Thanks YuLing and
> XiaoWei)
> > - Update test framework scripts to support different releases we
> will make. (Thanks YuLing).
> > - Create scripts under utilities to facilate our work (Thanks
> XiaoWei)
> > - Update test framework document to v0.9 (Thanks YuLing)
> >
> > Test case contribution and test execution are welcome!
> >
> > The README in the release package describes how to use the framework,
> how to contribute more tests to the framework in a brief. For more
> details, please refer to Framework_Specification.htm document under doc
> directory in the release package.
> >
> > The goal of this project is to create conformance test suite for SAF
> spec, including AIS A.01.01, AIS B.01.01, HPI A.01.01 and HPI B.01.01
> spec.
> >
> > Any more comments, please discuss in the following mailing list:
> > saftest-devel at lists.sourceforge.net (For the whole project)
> saftest-hpi-devel at lists.sourceforge.net (For HPI part)
> saftest-ais-devel at lists.sourceforge.net (For AIS part)
> >
> > The webpage of this project can be found here:
> http://saftest.sourceforge.net/
> >
> > Thanks,
> > Crystal
> >
> > * This is my personal opinion *
> >
> >
> >
> > -------------------------------------------------------
> > This SF.Net email is sponsored by: IntelliVIEW -- Interactive
> Reporting
> > Tool for open source databases. Create drag-&-drop reports. Save time
> > by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> > Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> > _______________________________________________
> > Saftest-devel mailing list
> > Saftest-devel at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/saftest-devel
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
> Tool for open source databases. Create drag-&-drop reports. Save time
> by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> _______________________________________________
> Saftest-ais-devel mailing list
> Saftest-ais-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/saftest-ais-devel
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
> Tool for open source databases. Create drag-&-drop reports. Save time
> by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> _______________________________________________
> Saftest-hpi-devel mailing list
> Saftest-hpi-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/saftest-hpi-devel
More information about the Openais
mailing list