Skip to content
Software Testing Survey
*Note: If you are part of an organization that includes multiple product groups, please complete for the product group with which you are most familiar.
Which of the following categories best describe the software that your organization builds?
eCommerce/Shopping
Finance/Banking
Social Media
Gaming
Media and Entertainment
Business Applications
Content Creation/Editing/Management
Data Management
Education
Development
Security
Network Management
Systems Management
Other (please specify)
How many
software engineers/developers
work at your organization?
0
1-5
6-25
26-50
51-100
101-1000
>1000
I don't know
On average, what fraction of a software engineer/developer's time is spent testing software within your organization?
0-5%
6-10%
11-25%
26-50%
51-75%
>75%
I don't know
How many full time
test/QA engineers
work in your organization?
0
1-5
6-25
26-50
51-100
101-1000
>1000
I don't know
How many
test automation engineers
work in your organization?
0
1-5
6-10
11-25
26-100
>100
I don't know
Which of the following testing approaches does your organization employ? [Select all that apply]
Manual functional testing for web applications
Automated functional testing for web applications
Manual functional testing for mobile applications
Automated functional testing for mobile applications
Load testing
Synthetic Transaction Monitoring
I don't know
Other (please specify)
Which of the following tools does your organization use for
functional testing of web applications
? [Select all that apply]
Selenium
Smartbear
Tricentis Tosca
Rainforest QA
Sauce Labs
Applause/uTest
Parasoft
HP UFT/LeanFT/Sprinter
IBM Rational
None of the above / I don't know
Other (please specify)
Approximately how much did your organization spend on
functional testing products and services for web applications
in 2016?
<$100
$100 - $1,000
$1,000 - $10,000
$10,000-$100,000
$100,000-$1,000,000
$1,000,000 or more
I don't know
Which of the following tools does your organization use for
functional testing of mobile applications
? [Select all that apply]
AWS Device Farm
Appium
Sauce Labs
Ranorex
Calabash
Robotium
UI Automator
Moneyrunner
Selendriod
Smartbear
Applause
Other (please specify)
Approximately how much did your organization spend on
functional testing products and services for mobile applications
in 2016?
<$100
$100 - $1,000
$1,000 - $10,000
$10,000-$100,000
$100,000-$1,000,000
$1,000,000 or more
I don't know
Which of the following tools does your organization use for
load testing
? [Select all that apply]
Smartbear
Blazemeter
Apica
Loadrunner
JMeter
Testplant/Eggplant
Loadimpact
Locust
Gatling
Other
Other (please specify)
Approximately how much did your organization spend on
load testing products and services
in 2016?
<$100
$100 - $1,000
$1,000 - $10,000
$10,000-$100,000
$100,000-$1,000,000
$1,000,000 or more
I don't know
Which of the following tools does your organization use for
synthetic transaction monitoring
? [Select all that apply]
New Relic
Zabbix
Solarwinds/Tracelytics
Pingdom
Dynatrace
AppDynamics
Catchpoint
Runscope
None
I don't know
Other (please specify)
Which of the following tools does your organization currently use to track bugs/defects? [Select all that apply]
Jira
Github
BugZilla
FogBugz
Spreadsheets
None
I don't know
Other (please specify)
How satisfied are you with the following aspects of the testing process at your company?
Very Dissatisfied
Dissatisfied
Neutral
Satisfied
Very Satisfied
N/A
Level of complexity to maintain test automation
Very Dissatisfied
Dissatisfied
Neutral
Satisfied
Very Satisfied
N/A
Fraction of developer time spent testing
Very Dissatisfied
Dissatisfied
Neutral
Satisfied
Very Satisfied
N/A
Level of complexity to automate testing
Very Dissatisfied
Dissatisfied
Neutral
Satisfied
Very Satisfied
N/A
Time spent documenting and maintaining test cases
Very Dissatisfied
Dissatisfied
Neutral
Satisfied
Very Satisfied
N/A
Ability replicate production in pre-production environment
Very Dissatisfied
Dissatisfied
Neutral
Satisfied
Very Satisfied
N/A
Functional/integration test coverage
Very Dissatisfied
Dissatisfied
Neutral
Satisfied
Very Satisfied
N/A
Finding defects before they hit production
Very Dissatisfied
Dissatisfied
Neutral
Satisfied
Very Satisfied
N/A
Cost of testing tools
Very Dissatisfied
Dissatisfied
Neutral
Satisfied
Very Satisfied
N/A
Bug reporting and tracking
Very Dissatisfied
Dissatisfied
Neutral
Satisfied
Very Satisfied
N/A
Unit test coverage
Very Dissatisfied
Dissatisfied
Neutral
Satisfied
Very Satisfied
N/A
Please indicate the extent to which you agree with the following statements.
1 - Strongly Disagree
2 - Disagree
3 - Neutral / NA
4 - Agree
5 - Strongly Agree
Our current testing toolset provides strong support for our application architecture.
1 - Strongly Disagree
2 - Disagree
3 - Neutral / NA
4 - Agree
5 - Strongly Agree
I am actively looking for new tools and technologies to improve testing
1 - Strongly Disagree
2 - Disagree
3 - Neutral / NA
4 - Agree
5 - Strongly Agree
I am concerned that defects in production are impacting the satisfaction of my customers.
1 - Strongly Disagree
2 - Disagree
3 - Neutral / NA
4 - Agree
5 - Strongly Agree
The output from my testing tools/process is sufficient for our engineers to validate and prioritize issues.
1 - Strongly Disagree
2 - Disagree
3 - Neutral / NA
4 - Agree
5 - Strongly Agree
I believe that the "state of the art" in the testing space has advanced as quickly as other aspects of software development.
1 - Strongly Disagree
2 - Disagree
3 - Neutral / NA
4 - Agree
5 - Strongly Agree
It is relatively easy to find engineers with test automation expertise as compared to other skill sets.
1 - Strongly Disagree
2 - Disagree
3 - Neutral / NA
4 - Agree
5 - Strongly Agree
Our current testing toolset is easily integrated with our release and deployment processes
1 - Strongly Disagree
2 - Disagree
3 - Neutral / NA
4 - Agree
5 - Strongly Agree
Our current testing toolset provides strong support for mobile
1 - Strongly Disagree
2 - Disagree
3 - Neutral / NA
4 - Agree
5 - Strongly Agree
We rely on users to find too many defects in production.
1 - Strongly Disagree
2 - Disagree
3 - Neutral / NA
4 - Agree
5 - Strongly Agree
Would you be willing to explore this topic with us further on a 15-minute call?
Yes
No
Would you like to be entered into our drawing to win a DJI Phantom 3 drone?
Yes
No
Please provide your email address. Note: We will only use it to notify the drone winner and/or to coordinate a follow-up call, if applicable.