**Before you sign with a product testing agency, know what you're** **getting into. Ask how they screen respondents for fraud, who owns the** **raw data, what's included in the quoted price versus billed as an** **add-on, and how comparable their methodology is across studies over** **time.** Those four alone eliminate most of the surprises that show up after a contract is signed. But a few more questions will help ensure a successful engagement and ongoing testing relationship. ## Why these questions matter when hiring a product testing agency Like any vendor, product testing agencies are good at selling their strengths and quiet about their limitations. The goal of this checklist isn't to feed your paranoia. It's to provide a rubric for evaluating vendors on the things that actually matter: data quality, process, cost, and rigor. How slick was their pitch? How glossy is their marketing packet? Neither make the cut. ## On data quality **1. How do you screen for fraudulent or low-quality respondents?** Bots, professional survey-takers, and disengaged panelists aren't a new problem in market research, but one that's more pervasive than ever. Ask specifically what's screened for—duplicate device fingerprints, response-time outliers, attention checks, open-ended response quality—and not just "we have quality controls." Vague answers tell you all you need to know: they're the same vague answers behind [a launch forecast quietly built on bad panel data](/content/blog/why-bad-panel-data-wrecks-launch-forecasts/index.html). **2. What's your panel's average response and completion rate?** Low engagement rates are a proxy for panel fatigue or low-quality espondents. Ask for a number, not a description. **3. Can you hit my specific audience, and at what incidence rate?** Narrow audiences (a specific usage occasion, a competitor's customer base, a niche dietary need) are the hardest tests on panel quality. Ask how these respondents would be sourced and how long it would take. ## On process and transparency **4. What visibility do I have into the study while it's fielding?** Some vendors deliver a black box—sign the contract, time passes, you'll get a deck. Others give real-time visibility into recruiting progress, response rates, and early data. If you're making time-sensitive decisions, know which one you're signing up for. **5. Who owns the data and the panel relationships?** If you switch vendors next year, do you keep your historical data and audience definitions, or does it live inside their proprietary system with no path to portability? **6. What's your standard turnaround, and what does rushing it cost?** Get real numbers for standard versus expedited timelines, and check that answer against [realistic timelines by method](/content/blog/how-long-does-product-testing-take/index.html) before you take a vendor's definition of "fast" at face value. ## On cost and scope **7. What's included in the quoted price, and what's billed separately?** Recruiting, fielding, analysis, and reporting are sometimes bundled and sometimes billed as separate line items. Ask this explicitly before you compare quotes against [typical IHUT and CLT cost ranges](/content/blog/ihut-clt-pricing-guide/index.html), or you'll end up comparing numbers that were never apples-to-apples. **8. What happens if the study needs to change mid-flight?** Business conditions change and new questions come up. Ask what it costs, in time and money, to adjust a screener or add a question after fielding has started. ## On methodology and consistency **9. Can you maintain consistent methodology and audience definitions** **across multiple studies?** If you're planning to test a concept, then a prototype, then an in-market read over the coming months, ask whether the vendor can keep some or all of the audience and protocol consistent across multiple projects, or whether you'll be starting each engagement from scratch. This is the difference between a set of comparable data points and three unrelated snapshots. **10. How do you handle the handoff from raw data to a decision my** **executive team will trust?** Ask to see a sample deliverable, not just a description of "robust analytics." The gap between a data file and a stakeholder-ready recommendation is often where the real value—or the real disappointment—shows up. ## Using this as a scorecard Print these ten questions and score each vendor on a simple scale rather than relying on gut feel after a sales call. And remember: - Vague answers to the fraud and screening questions are the single biggest red flag on this list. - A vendor who can't explain what's bundled into a quote is a vendor whose final invoice will surprise you. - If continuity across studies matters to your roadmap, weight question 9 heavily—it's the one that determines whether your research compounds into an asset or stays a series of one-off projects. ## Beyond the checklist This list assumes you've already settled on bringing in outside help. If you're still weighing whether this particular project should go to an agency at all—versus staying in-house, or running it through an integrated platform— [In-House vs. Agency Product Testing](/content/blog/in-house-vs-agency-product-testing/index.html) writes through that decision first. Ready to skip the vendor scorecard? Highlight's proprietary Highlighter Community, in-house logistics, and fraud-screening standards are standing infrastructure, and every study runs with the real-time visibility question 4 describes—not a black box—so the answers to most of these ten questions are already built in. [Explore Highlight's product testing services](/content/use-case/product-testing-services/index.html) or [book a strategy session](/content/book-a-test-strategy-session-highlight/index.html). **Related reading:** [How to evaluate research software](/content/blog/how-to-evaluate-research-software/index.html) · [How to Choose a Consumer Product Testing Company](/content/blog/product-testing-companies/index.html) · [How to Stop Survey Fraud in Market Research](/content/blog/survey-fraud/index.html).