Tools Available - 3
TestAgent - Capture/playback tool for user acceptance testing from Strenuus, LLC. Key features besides capture/playback include automatically detecting and capturing standard and custom content errors. Reports information needed to troubleshoot problems. Enables 'Persistent Acceptance Testing' that activates tests each time a web application is used.
MITS.GUI - Unique test automation tool from Omsphere LLC; has an intelligent state machine engine that makes real-time decisions for navigating through the GUI portion of an application. It can test thousands of test scenarios without use of any scripts. Allows creation of completely new test scenarios without ever having performed that test before, all without changing tool, testware architecture (object names, screen names, etc), or logic associated with the engine. Testers enter test data into a spreadsheet used to populate objects that appear for the particular test scenario defined.
Badboy - Tool from Bradley Software to aid in building and testing dynamic web based applications. Combines sophisticated capture/replay ability with performance testing and regression features. Free for most uses; source code avalable.
SAMIE - Free tool designed for QA engineers - 'Simple Automated Module For Internet Explorer'. Perl module that allows a user to automate use of IE via Perl scripts; Written in ActivePerl, allowing inheritance of all Perl functionality including regular expressions, Perl dbi database access, many Perl cpan library functions. Uses IE's built in COM object which provides a reference to the DOM for each browser window or frame. Easy development and maintenance - no need to keep track of GUI maps for each window. For Windows.
PAMIE - Free open-source 'Python Automated Module For Internet Explorer' Allows control of an instance of MSIE and access to it's methods though OLE automation . Utilizes Collections, Methods, Events and Properties exposed by the DHTML Object Model.
PureTest - Free tool from Minq Software AB, includes an HTTP Recorder and Web Crawler. Create scenarios using the point and click interface. Includes a scenario debugger including single step, break points and response introspection. Supports HTTPS/SSL, dynamic Web applications, data driven scenarios, and parsing of response codes or parsing page content for expected or unexpected strings. Includes a Task API for building custom test tasks. The Web Crawler is useful for verifying consistency of a static web structure, reporting various metrics, broken links and the structure of the crawled web. Multi-platform - written in Java.
Solex - Web application testing tool built as a plug-in for the Eclipse IDE (an open, extensible IDE). Records HTTP messages by acting as a Web proxy; recorded sessions can be saved as XML and reopened later. HTTP requests and responses are fully displayed in order to inspect and customize their content. Allows the attachment of extraction or replacement rules to any HTTP message content, and assertions to responses in order to validate a scenario during its playback.
QA Wizard - Automated functional web test tool from Seapine Software. Advanced object binding reduces script changes when Web-based apps change. Next-generation scripting language eliminates problems created by syntax or other language errors. Includes capability for automated scripting, allowing creation of more scripts in less time. Supports unlimited set of ODBC-compatible data sources as well as MS Excel, tab/comma delimited file formats, and more. Free Demo and Test Script available. For Windows platforms.
Software Testing Training
Our Software Testing Partner
offshore Software testing
Friday, January 9, 2009
Tools Available - 2
Tools Available - 2
WebInject - Open source tool in PERL for automated testing of web applications and services. Can be used to unit test any individual component with an HTTP interface (JSP, ASP, CGI, PHP, servlets, HTML forms, etc.) or it can be used to create a suite of HTTP level functional or regression tests.
Site Test Center - Functional and performance test tool from Alliance Software Engineering. Has an XML-based scripting capability to enable modifying captured scripts or creating new scripts. Utilizes a distributed testing model and consists of three parts: STC Administrator, STC Master and STC Master Service.
jWebUnit - Open source Java framework that facilitates creation of acceptance tests for web applications. Provides a high-level API for navigating a web application combined with a set of assertions to verify the application's correctness including navigation via links, form entry and submission, validation of table contents, and other typical business web application features. Utilizes HttpUnit behind the scenes. The simple navigation methods and ready-to-use assertions allow for more rapid test creation than using only JUnit and HttpUnit.
SimpleTest - Open source unit testing framework which aims to be a complete PHP developer test solution. Includes all of the typical functions that would be expected from JUnit and the PHPUnit ports, but also adds mock objects; has some JWebUnit functionality as well. This includes web page navigation, cookie testing and form submission.
WinTask - Macro recorder from TaskWare, automates repetitive tasks for Web site testing (and standard Windows applications), with its HTML objects recognition. Includes capability to expand scope of macros by editing and adding loops, branching statements, etc. (300+ commands); ensure robustness of scripts with Synchronization commands. Includes a WinTask Scheduler.
TestCaseMaker/Runner - Test case document driven functional test tool for web applications from Agile Web Development. Maker creates test case documents, and Runner executes the test case document; test case documents are always synchronized with the application. Free including source code.
Canoo WebTest - Free Java Open Source tool for automatic functional testing of web applications. XML-based test script code is editable with user's preferred XML editor; until recording capabilities are added, scripts have to be developed manually. Can group tests into a testsuite that again can be part of a bigger testsuite. Test results are reported in either plain text or XML format for later presentation via XSLT. Standard reporting XSLT stylesheets included, and can be adapted to any reporting style or requirements.
TestSmith - Functional/Regression test tool from Quality Forge. Includes an Intelligent, HTML/DOM-Aware and Object Mode Recording Engine, and a Data-Driven, Adaptable and Multi-Threaded Playback Engine. Handles Applets, Flash, Active-X controls, animated bitmaps, etc. Controls are recorded as individual objects independent of screen positions or resolution; playback window/size can be different than in capture. Special validation points, such as bitmap or text matching, can be inserted during a recording, but all recorded items are validated and logged 'on the fly'. Fuzzy matching capabilities. Editable scripts can be recorded in SmithSript language or in Java, C++ or C++/MFC. 90-day evaluation copy available.
Software Testing Training
Our Software Testing Partner
offshore Software testing
WebInject - Open source tool in PERL for automated testing of web applications and services. Can be used to unit test any individual component with an HTTP interface (JSP, ASP, CGI, PHP, servlets, HTML forms, etc.) or it can be used to create a suite of HTTP level functional or regression tests.
Site Test Center - Functional and performance test tool from Alliance Software Engineering. Has an XML-based scripting capability to enable modifying captured scripts or creating new scripts. Utilizes a distributed testing model and consists of three parts: STC Administrator, STC Master and STC Master Service.
jWebUnit - Open source Java framework that facilitates creation of acceptance tests for web applications. Provides a high-level API for navigating a web application combined with a set of assertions to verify the application's correctness including navigation via links, form entry and submission, validation of table contents, and other typical business web application features. Utilizes HttpUnit behind the scenes. The simple navigation methods and ready-to-use assertions allow for more rapid test creation than using only JUnit and HttpUnit.
SimpleTest - Open source unit testing framework which aims to be a complete PHP developer test solution. Includes all of the typical functions that would be expected from JUnit and the PHPUnit ports, but also adds mock objects; has some JWebUnit functionality as well. This includes web page navigation, cookie testing and form submission.
WinTask - Macro recorder from TaskWare, automates repetitive tasks for Web site testing (and standard Windows applications), with its HTML objects recognition. Includes capability to expand scope of macros by editing and adding loops, branching statements, etc. (300+ commands); ensure robustness of scripts with Synchronization commands. Includes a WinTask Scheduler.
TestCaseMaker/Runner - Test case document driven functional test tool for web applications from Agile Web Development. Maker creates test case documents, and Runner executes the test case document; test case documents are always synchronized with the application. Free including source code.
Canoo WebTest - Free Java Open Source tool for automatic functional testing of web applications. XML-based test script code is editable with user's preferred XML editor; until recording capabilities are added, scripts have to be developed manually. Can group tests into a testsuite that again can be part of a bigger testsuite. Test results are reported in either plain text or XML format for later presentation via XSLT. Standard reporting XSLT stylesheets included, and can be adapted to any reporting style or requirements.
TestSmith - Functional/Regression test tool from Quality Forge. Includes an Intelligent, HTML/DOM-Aware and Object Mode Recording Engine, and a Data-Driven, Adaptable and Multi-Threaded Playback Engine. Handles Applets, Flash, Active-X controls, animated bitmaps, etc. Controls are recorded as individual objects independent of screen positions or resolution; playback window/size can be different than in capture. Special validation points, such as bitmap or text matching, can be inserted during a recording, but all recorded items are validated and logged 'on the fly'. Fuzzy matching capabilities. Editable scripts can be recorded in SmithSript language or in Java, C++ or C++/MFC. 90-day evaluation copy available.
Software Testing Training
Our Software Testing Partner
offshore Software testing
Tools Available - 1
Tools Available - 1
IeUnit - IeUnit is an open-source simple framework to test logical behaviors of web pages, released under IBM's Common Public License. It helps users to create, organize and execute functional unit tests. Includes a test runner with GUI interface. Implemented in JavaScript for the Windows XP platform with Internet Explorer.
QEngine Web Test Studio - Web functional test tool from AdventNet. Scripting uses Jython; records using page elements controls symbolically rather than with raw screen coordinate. Secure recording on password fields; data-driven Test wizard to fetch script data from external source; provision to add GUI, Database and File checkpoints and verify database tables, files, page titles and HTML element properties. Supports keyword-driven testing, built-in exception handling and reporting facility. Works with a variety of browsers and OS's. Free and professional versions available.
AppPerfect DevSuite - Suite of testing, tuning, and monitoring products from AppPefect Corp. that includes a web functional testing module. Records browser interaction by element instead of screen co-ordinates. Supports handling dynamic content created by JavaScript; supports ASP, JSP, HTML, cookies, SSL. For Windows and MSIE; integrates with a variety of IDE's.
JStudio SiteWalker - Test tool from Jarsch Software Studio allows capture/replay recording; fail definitions can be specified for each step of the automated workflow via JavaScript. JavaScript's Document Object Model enables full access to all document elements. Test data from any database or Excel spreadsheet can be mapped to enter values automatically into HTML form controls. HTML-based test result reports can be generated. Shareware for Windows/MSIE.
Test Complete Enterprise - Automated test tool from AutomatedQA Corp. includes web functional testing capabilities. Works with Internet Explorer. QEngine - Test tool from AdventNet enables functional testing of Web sites and Web-based applications. Record and playback capability; automatic recording of any Web browser events and translates into an Python editable scripts. Includes Script Editor, Application Map Editor to view and edit the map object properties. Supports multiple OS's and browsers.
actiWate - Java-based Web application testing environment from Actimind Inc. Advanced framework for writing test scripts in Java (similar to open-source frameworks like HttpUnit, HtmlUnit etc. but with extended API), and Test Writing Assistant - Web browser plug-in module to assist the test writing process. Freeware.
KUMO Editor - Toolset from Softmorning LTD for creation and editing of web macros and automated web tests. Includes syntax-coloring editor with intellisense, autocomplete, run-time debugging features. Macro recorder transforms any click to a C# directive. Page objects navigator allows browsing of hierarchy of web objects in a page. Enables creation of scenarios from spreadsheets; and loop, retry on error, robust handling of page modifications. Can export created .DLL and .EXE files to enable running web macros on demand and integration into other software frameworks. Multilingual for Asian, eastern and western European languages.
Software Testing Training
Our Software Testing Partner
offshore Software testing
IeUnit - IeUnit is an open-source simple framework to test logical behaviors of web pages, released under IBM's Common Public License. It helps users to create, organize and execute functional unit tests. Includes a test runner with GUI interface. Implemented in JavaScript for the Windows XP platform with Internet Explorer.
QEngine Web Test Studio - Web functional test tool from AdventNet. Scripting uses Jython; records using page elements controls symbolically rather than with raw screen coordinate. Secure recording on password fields; data-driven Test wizard to fetch script data from external source; provision to add GUI, Database and File checkpoints and verify database tables, files, page titles and HTML element properties. Supports keyword-driven testing, built-in exception handling and reporting facility. Works with a variety of browsers and OS's. Free and professional versions available.
AppPerfect DevSuite - Suite of testing, tuning, and monitoring products from AppPefect Corp. that includes a web functional testing module. Records browser interaction by element instead of screen co-ordinates. Supports handling dynamic content created by JavaScript; supports ASP, JSP, HTML, cookies, SSL. For Windows and MSIE; integrates with a variety of IDE's.
JStudio SiteWalker - Test tool from Jarsch Software Studio allows capture/replay recording; fail definitions can be specified for each step of the automated workflow via JavaScript. JavaScript's Document Object Model enables full access to all document elements. Test data from any database or Excel spreadsheet can be mapped to enter values automatically into HTML form controls. HTML-based test result reports can be generated. Shareware for Windows/MSIE.
Test Complete Enterprise - Automated test tool from AutomatedQA Corp. includes web functional testing capabilities. Works with Internet Explorer. QEngine - Test tool from AdventNet enables functional testing of Web sites and Web-based applications. Record and playback capability; automatic recording of any Web browser events and translates into an Python editable scripts. Includes Script Editor, Application Map Editor to view and edit the map object properties. Supports multiple OS's and browsers.
actiWate - Java-based Web application testing environment from Actimind Inc. Advanced framework for writing test scripts in Java (similar to open-source frameworks like HttpUnit, HtmlUnit etc. but with extended API), and Test Writing Assistant - Web browser plug-in module to assist the test writing process. Freeware.
KUMO Editor - Toolset from Softmorning LTD for creation and editing of web macros and automated web tests. Includes syntax-coloring editor with intellisense, autocomplete, run-time debugging features. Macro recorder transforms any click to a C# directive. Page objects navigator allows browsing of hierarchy of web objects in a page. Enables creation of scenarios from spreadsheets; and loop, retry on error, robust handling of page modifications. Can export created .DLL and .EXE files to enable running web macros on demand and integration into other software frameworks. Multilingual for Asian, eastern and western European languages.
Software Testing Training
Our Software Testing Partner
offshore Software testing
The Goals of Bug Writing
The Goals of Bug Writing
As with all other forms of writing, it‘s important to remember who your audience is. Usually, the bug will be written for developers and for other QA engineers. Know also that other departments may need to understand the bug: marketing, tech support, etc. You can assume that everyone who may read the bug will have some understanding about the product which the bug is logged against and the technology used. They may not, however, know much about your specific test area.
1. Eliminate basic questions that a Development Engineer might have by including essential information.
2. Understand your audience: make steps understandable for other departments (tech support, marketing) or other testers not in your area.
3. Make searching easier for yourself and others.
4. Write in a way that demonstrates you‘ve done necessary isolation.
Title
Keep the title short and sweet. It should be as explicit as possible in as few words as possible. Think again about what the title will be used for and who might use it. A QA engineer might be looking to match their bug against what‘s already logged and needs a quick way to scan through the titles. Distill each bug to its crucial elements and put that in the title.
1. Try to write in a cause-and-effect manner (―When A is done, B happens,‖ or ―B happens when A is done.‖)
2. Avoid ambiguous wording, such as ―feature is broken/incorrect behavior/does not work,‖ etc. Instead, say precisely how it‘s broken, incorrect, or not working.
3. Use keywords that will make searching for the bug easier for yourself or someone else looking for duplicate bugs. Avoid using jargon, slang, or vocabulary that is too specific to your area.
4. When an assert appears in your bug, include the assert or a portion of the assert in the title(e.g., ―Assert, ‗index < fLength‘ when pasting text into very small text frame‖).
Description
This is where all of the information, the body of the bug, resides: Steps to Reproduce, Actual Results, Expected Results, and any other helpful or vital information regarding the bug.
Steps to Reproduce Bug reports usually suffer from two deficiencies:
1. Too many steps, often poorly organized. Having too many steps in a bug report makes it difficult to read and understand, especially given the confined viewing area in Vantive.
2. Too little information in a bug often leads to unnecessary extra effort and time. Often a bug will be sent back by the engineer as Cannot Reproduce as a result of the inability to follow poorly constructed steps in a valid bug. This can also indicate that you haven‘t completed enough isolation steps.
Essential information to include in Steps to Reproduce:
1. Setup variables: Indicate which printers, fonts, or drivers are necessary to reproduce this bug. Indicate the working OS, if it‘s essential information for reproducing this bug.
2. Environmental variables. For example, indicate if you are working in application or document mode.
Software Testing Training
Our Software Testing Partner
offshore Software testing
As with all other forms of writing, it‘s important to remember who your audience is. Usually, the bug will be written for developers and for other QA engineers. Know also that other departments may need to understand the bug: marketing, tech support, etc. You can assume that everyone who may read the bug will have some understanding about the product which the bug is logged against and the technology used. They may not, however, know much about your specific test area.
1. Eliminate basic questions that a Development Engineer might have by including essential information.
2. Understand your audience: make steps understandable for other departments (tech support, marketing) or other testers not in your area.
3. Make searching easier for yourself and others.
4. Write in a way that demonstrates you‘ve done necessary isolation.
Title
Keep the title short and sweet. It should be as explicit as possible in as few words as possible. Think again about what the title will be used for and who might use it. A QA engineer might be looking to match their bug against what‘s already logged and needs a quick way to scan through the titles. Distill each bug to its crucial elements and put that in the title.
1. Try to write in a cause-and-effect manner (―When A is done, B happens,‖ or ―B happens when A is done.‖)
2. Avoid ambiguous wording, such as ―feature is broken/incorrect behavior/does not work,‖ etc. Instead, say precisely how it‘s broken, incorrect, or not working.
3. Use keywords that will make searching for the bug easier for yourself or someone else looking for duplicate bugs. Avoid using jargon, slang, or vocabulary that is too specific to your area.
4. When an assert appears in your bug, include the assert or a portion of the assert in the title(e.g., ―Assert, ‗index < fLength‘ when pasting text into very small text frame‖).
Description
This is where all of the information, the body of the bug, resides: Steps to Reproduce, Actual Results, Expected Results, and any other helpful or vital information regarding the bug.
Steps to Reproduce Bug reports usually suffer from two deficiencies:
1. Too many steps, often poorly organized. Having too many steps in a bug report makes it difficult to read and understand, especially given the confined viewing area in Vantive.
2. Too little information in a bug often leads to unnecessary extra effort and time. Often a bug will be sent back by the engineer as Cannot Reproduce as a result of the inability to follow poorly constructed steps in a valid bug. This can also indicate that you haven‘t completed enough isolation steps.
Essential information to include in Steps to Reproduce:
1. Setup variables: Indicate which printers, fonts, or drivers are necessary to reproduce this bug. Indicate the working OS, if it‘s essential information for reproducing this bug.
2. Environmental variables. For example, indicate if you are working in application or document mode.
Software Testing Training
Our Software Testing Partner
offshore Software testing
Test Management System
Test Management System
Test design and procedure development
At the test design stage, testers will create a description of each test, document its scope and objective, and include any information that helps illustrate the purpose of a specific test such as requirements documents, functional specifications, etc. During the test development phase, testers will document detailed test execution steps and define the expected results for each step. The test management system will help in defining and documenting test cases by providing standard Web-based, pre-formatted template forms with fields based on the product and component information for editing and creating test cases. These get posted to the centralized database. This enables standardization and consistency across the testing team. It will also help in linking to the requirements specification to ensure traceability and test coverage. The test case may have been created due to a known defect and gets an association created with that defect. We can define the sequence in which test cases should be executed. This may be based on functional dependencies or some other factors like risk and other priority.
Organization
To verify application functionality and usability, tests have to realistically emulate end-user behavior. To achieve this, test execution should follow predefined logic, such as running certain tests after other tests have passed, failed, or been completed. For example, a user logs into the system, enters a new order and then exits the system. To emulate this simple business process, it makes sense to run the tests following the exact same sequence: log in, insert order, log out. The execution logic rules should be set prior to executing the actual tests.
Review
Once the test cases have been created, we can get them reviewed by required team members and customers. It is easy to communicate the test cases to the team because of the Web interface. This will verify the test cases developed by the test team and improve them further if required before actual testing begins.
Execution
The test cases can be accessed from any computer over the intranet/Internet, depending on how the test management tool is deployed. The test management system will help in locating a test case and provide a Web interface to process the test case. As the test is processed, the tester can immediately log the actual results along with pass/fail results and additional comments. The Web-based process supports parallel execution of test cases by many team members, which is not possible with a single flat file that gets "routed" around. If the test ever fails, it has an associated defect number which the tester can look at to see if a previous defect report should be opened, or a new one created. This helps to ensure that nothing falls through the cracks.
Maintenance
The test management system will maintain an accurate history of each run, including execution configuration, date and time of run, who ran the test, and any defects that were uncovered during the run.
Defect management
When a test case fails, the tester can enter the ID of the defect that caused the case to fail. The defect is, of course, inserted into the defect tracking system. The defect can be linked with the test cases. This will provide information to reproduce and analyze the defects.
Software Testing Training
Our Software Testing Partner
offshore Software testing
Test design and procedure development
At the test design stage, testers will create a description of each test, document its scope and objective, and include any information that helps illustrate the purpose of a specific test such as requirements documents, functional specifications, etc. During the test development phase, testers will document detailed test execution steps and define the expected results for each step. The test management system will help in defining and documenting test cases by providing standard Web-based, pre-formatted template forms with fields based on the product and component information for editing and creating test cases. These get posted to the centralized database. This enables standardization and consistency across the testing team. It will also help in linking to the requirements specification to ensure traceability and test coverage. The test case may have been created due to a known defect and gets an association created with that defect. We can define the sequence in which test cases should be executed. This may be based on functional dependencies or some other factors like risk and other priority.
Organization
To verify application functionality and usability, tests have to realistically emulate end-user behavior. To achieve this, test execution should follow predefined logic, such as running certain tests after other tests have passed, failed, or been completed. For example, a user logs into the system, enters a new order and then exits the system. To emulate this simple business process, it makes sense to run the tests following the exact same sequence: log in, insert order, log out. The execution logic rules should be set prior to executing the actual tests.
Review
Once the test cases have been created, we can get them reviewed by required team members and customers. It is easy to communicate the test cases to the team because of the Web interface. This will verify the test cases developed by the test team and improve them further if required before actual testing begins.
Execution
The test cases can be accessed from any computer over the intranet/Internet, depending on how the test management tool is deployed. The test management system will help in locating a test case and provide a Web interface to process the test case. As the test is processed, the tester can immediately log the actual results along with pass/fail results and additional comments. The Web-based process supports parallel execution of test cases by many team members, which is not possible with a single flat file that gets "routed" around. If the test ever fails, it has an associated defect number which the tester can look at to see if a previous defect report should be opened, or a new one created. This helps to ensure that nothing falls through the cracks.
Maintenance
The test management system will maintain an accurate history of each run, including execution configuration, date and time of run, who ran the test, and any defects that were uncovered during the run.
Defect management
When a test case fails, the tester can enter the ID of the defect that caused the case to fail. The defect is, of course, inserted into the defect tracking system. The defect can be linked with the test cases. This will provide information to reproduce and analyze the defects.
Software Testing Training
Our Software Testing Partner
offshore Software testing
Bug Logging / Writing / Reporting
Bug Logging / Writing / Reporting
The first aim of a bug report is to let the programmer see the failure with their own eyes. If you can't be with them to make it fail in front of them, give them detailed instructions so that they can make it fail for themselves.
In case the first aim doesn't succeed, and the programmer can't see it failing themselves, the second aim of a bug report is to describe what went wrong. Describe everything in detail.State what you saw, and also state what you expected to see. Write down the error messages, especially if they have numbers in.
When your computer does something unexpected, freeze. Do nothing until you're calm, and don't do anything that you think might be dangerous. By all means try to diagnose the fault yourself if you think you can, but if you do, you should still report the symptoms as well.
Be ready to provide extra information if the programmer needs it. If they didn't need it, they wouldn't be asking for it. They aren't being deliberately awkward. Have version numbers at your fingertips, because they will probably be needed. Write clearly. Say what you mean, and make sure it can't be misinterpreted. Above all, be precise. Programmers like precision. Useful bug reports are ones that get bugs fixed. A useful bug report normally has two qualities:
1. Reproducible. If an engineer can't see it or conclusively prove that it exists, the engineer will probably stamp it WORKSFORME or INVALID, and move on to the next bug. Every relevant detail you can provide helps.
2. Specific. The quicker the engineer can isolate the issue to a specific problem, the more likely it'll be expediently fixed. If you're crashing on a site, please take the time to isolate what on the page is triggering the crash, and include it as an HTML snippet in the bug report if possible. (Specific bugs have the added bonus of remaining relevant when an engineer actually gets to them; in a rapidly changing web, a bug report of "foo.com crashes my browser" becomes meaningless after the site experiences a half-dozen redesigns and hundreds of content changes.)
Software Testing Training
Our Software Testing Partner
offshore Software testing
The first aim of a bug report is to let the programmer see the failure with their own eyes. If you can't be with them to make it fail in front of them, give them detailed instructions so that they can make it fail for themselves.
In case the first aim doesn't succeed, and the programmer can't see it failing themselves, the second aim of a bug report is to describe what went wrong. Describe everything in detail.State what you saw, and also state what you expected to see. Write down the error messages, especially if they have numbers in.
When your computer does something unexpected, freeze. Do nothing until you're calm, and don't do anything that you think might be dangerous. By all means try to diagnose the fault yourself if you think you can, but if you do, you should still report the symptoms as well.
Be ready to provide extra information if the programmer needs it. If they didn't need it, they wouldn't be asking for it. They aren't being deliberately awkward. Have version numbers at your fingertips, because they will probably be needed. Write clearly. Say what you mean, and make sure it can't be misinterpreted. Above all, be precise. Programmers like precision. Useful bug reports are ones that get bugs fixed. A useful bug report normally has two qualities:
1. Reproducible. If an engineer can't see it or conclusively prove that it exists, the engineer will probably stamp it WORKSFORME or INVALID, and move on to the next bug. Every relevant detail you can provide helps.
2. Specific. The quicker the engineer can isolate the issue to a specific problem, the more likely it'll be expediently fixed. If you're crashing on a site, please take the time to isolate what on the page is triggering the crash, and include it as an HTML snippet in the bug report if possible. (Specific bugs have the added bonus of remaining relevant when an engineer actually gets to them; in a rapidly changing web, a bug report of "foo.com crashes my browser" becomes meaningless after the site experiences a half-dozen redesigns and hundreds of content changes.)
Software Testing Training
Our Software Testing Partner
offshore Software testing
Automated Testing - Manual Vs Automation
Automated Testing - Manual Vs Automation
If you‘re only going to run the test one or two times or the test is really expensive to automation, it is most likely a manual test. But then again, what good is saying ―use common sense‖ when you need to come up with deterministic set of guidelines on how and when to automate?
Pros of Automation
If you have to run a set of tests repeatedly, automation is a huge win for you
It gives you the ability to run automation against code that frequently changes to catch regressions in a timely manner
It gives you the ability to run automation in mainstream scenarios to catch regressions in a timely manner
Aids in testing a large test matrix (different languages on different OS platforms).
Automated tests can be run at the same time on different machines, whereas the manual tests would have to be run sequentially.
Cons of Automation
It costs more to automate.
Writing the test cases and writing or configuring the automate framework you‘re using costs more initially than running the test manually.
Can‘t automate visual references, for example, if you can‘t tell the font color via code or the automation tool, it is a manual test.
Pros of Manual
If the test case only runs twice a coding milestone, it most likely should be a manual test. Less cost than automating it.
It allows the tester to perform more ad-hoc (random testing). More bugs are found via ad-hoc than via automation. And, the more time a tester spends playing with the feature, the greater the odds of finding real user bugs.
Cons of Manual
1. Running tests manually can be very time consuming
2. Each time there is a new build, the tester must rerun all required tests - which after a while would become very mundane and tiresome.
Other deciding factors
1. What you automate depends on the tools you use. If the tools have any limitations, those tests are manual.
2. Is the return on investment worth automating?
3. Is what you get out of automation worth the cost of setting up and supporting the test cases, the automation framework, and the system that runs the test cases?
Criteria for automating
There are two sets of questions to determine whether automation is right for your test case: Is this test scenario automatable?
1. Yes, and it will cost a little
2. Yes, but it will cost a lot
3. No, it is no possible to automate
How important is this test scenario?
1. I must absolutely test this scenario whenever possible
2. I need to test this scenario regularly
3. I only need to test this scenario once in a while
If you answered #1 to both questions – definitely automate that test
If you answered #1 or #2 to both questions – you should automate that test
If you answered #2 to both questions – you need to consider if it is really worth the investment to automate
What happens if you can‘t automate?
Let‘s say that you have a test that you absolutely need to run whenever possible, but it isn‘t possible to automate. Your options are
Reevaluate – do I really need to run this test this often?
What‘s the cost of doing this test manually?
Look for new testing tools
Consider test hooks
Software Testing Training
Our Software Testing Partner
offshore Software testing
If you‘re only going to run the test one or two times or the test is really expensive to automation, it is most likely a manual test. But then again, what good is saying ―use common sense‖ when you need to come up with deterministic set of guidelines on how and when to automate?
Pros of Automation
If you have to run a set of tests repeatedly, automation is a huge win for you
It gives you the ability to run automation against code that frequently changes to catch regressions in a timely manner
It gives you the ability to run automation in mainstream scenarios to catch regressions in a timely manner
Aids in testing a large test matrix (different languages on different OS platforms).
Automated tests can be run at the same time on different machines, whereas the manual tests would have to be run sequentially.
Cons of Automation
It costs more to automate.
Writing the test cases and writing or configuring the automate framework you‘re using costs more initially than running the test manually.
Can‘t automate visual references, for example, if you can‘t tell the font color via code or the automation tool, it is a manual test.
Pros of Manual
If the test case only runs twice a coding milestone, it most likely should be a manual test. Less cost than automating it.
It allows the tester to perform more ad-hoc (random testing). More bugs are found via ad-hoc than via automation. And, the more time a tester spends playing with the feature, the greater the odds of finding real user bugs.
Cons of Manual
1. Running tests manually can be very time consuming
2. Each time there is a new build, the tester must rerun all required tests - which after a while would become very mundane and tiresome.
Other deciding factors
1. What you automate depends on the tools you use. If the tools have any limitations, those tests are manual.
2. Is the return on investment worth automating?
3. Is what you get out of automation worth the cost of setting up and supporting the test cases, the automation framework, and the system that runs the test cases?
Criteria for automating
There are two sets of questions to determine whether automation is right for your test case: Is this test scenario automatable?
1. Yes, and it will cost a little
2. Yes, but it will cost a lot
3. No, it is no possible to automate
How important is this test scenario?
1. I must absolutely test this scenario whenever possible
2. I need to test this scenario regularly
3. I only need to test this scenario once in a while
If you answered #1 to both questions – definitely automate that test
If you answered #1 or #2 to both questions – you should automate that test
If you answered #2 to both questions – you need to consider if it is really worth the investment to automate
What happens if you can‘t automate?
Let‘s say that you have a test that you absolutely need to run whenever possible, but it isn‘t possible to automate. Your options are
Reevaluate – do I really need to run this test this often?
What‘s the cost of doing this test manually?
Look for new testing tools
Consider test hooks
Software Testing Training
Our Software Testing Partner
offshore Software testing
Subscribe to:
Posts (Atom)