Managing Outsourced Testing----(2)question candidates

Now that you have your candidates, it is time to question, question and question. This is a key step in determining which lab to use. Here are some key points to get answered:

QUESTION

• Ask for the following sample deliverables:

  Testplan and testcases

  Defect reports

  Project summaries

  Sample deliverable output is critical. When work is being done offsite, the written and verbal communication takes on greater importance. Face to face meetings will be few and far between. Look for items such as clearly detailed defect reports, test cases with defined outcomes and very detailed project summaries.

• Ask about hardware / software

  Will the project share machines with others?

  What is the availability of project-specific equipment?

  If using a high volume test lab, chances are they share equipment across projects. It is in your best interest to find this out. If there are going to be issues, find out if the test lab will lease whatever equipment needed.

• How much hardware \ software do they have?

  Get an inventory of hardware and software. Any reputable lab should have all the latest equipment. Additionally labs should have a good supply of older machines that still proliferate in the marketplace. The goal is to get as close as possible to the machine class the target audience is using. Look to get a list of the following:

  CPU’s

  Printers

  MODEMS

Graphics cards

Network Components.

• Is their hardware organized and available?

  A lab may have a lot of equipment in inventory, but how well organized is it? In some cases I have seen labs with parts everywhere, and machines split between labs. Ask if machines travel from lab to lab and how they keep track of the components. A good sample question may be, If they had to find a video card how would they go about doing it?

• Do they use out of the box machines?

  Some labs add software to machines, which may be native to their company. This software may mask problems, which an average at home user would not have. A good way around this is find out if the lab has an “out of the box” machine. These machines are ones which consumers get directly from computer manufacturers and are free from company specific software. Remember, not every machine is blessed with spreadsheets and word processors. If there is DLL the software requires from a popular application or operating system component, then the home consumer may crash and burn. If testing a widely used product, this sort of testing is mandatory.

• Do the testers maintain the machines or does an I.T. staff?

  If testers are always configuring machines, then they do not have time to test. If the test lab has an I.T. squad, they can keep the machines up and running while the testers are focused on testing. An I.T. team can also be quicker in turning around a machine.

Ask about personnel

View tester resumes

• What are the testers Lab Technology cumulative experiences?

  Be sure to see the resume of any prospective tester. The sales staff will promise all kinds of skills but the testing is still being done by individuals. Review reports and project summaries created by the testers. Look for similar testing experience or products they have tested.

• Are the testers shared across projects?

  If the lab is to be used sporadically, the known testers may not be available at a later date. Find out if this is the case and plan accordingly. Once you locate a good tester request that person for future projects. In fact, ask for particular testers as part of the contract negotiations.

• Will they sign non-disclosures?
• Can testers be bonded?

  Since work is done offsite or with multiple clients a non-disclosure is always advised. If your software is financially sensitive be sure to get everyone involved on your projects bonded. This can be another contract point. Don’t lose good testers due to bonding issues.

• Do they have Project Management skills?

  Someone in the testlab will be responsible for coordinating the test work and communications with your company. Find out if the proposed lead has any of this experience. It makes communication flow a lot easier.

• Have they worked on similar project technologies?

  Have the testers worked on similar projects? This can be a big help, especially if the testers in your organization are not familiar with the new technologies.