Project Quant: Patch Management Process Survey

Thank you for participating in the Project Quant patch management survey. The goal of this survey is to gain a better understanding of current real-world patch management processes. Future surveys will help us validate the processes and specific patch management metrics developed as part of the project. All questions are optional, but the more details you provide the more accurate the eventual model will be. All results will be made publicly available (email addresses and personally or corporate identifiable information will be kept confidential).

1. Please provide the following size information on your organization. This will help us map processes based on the scale of an organization (rough estimates are fine):

Please provide the following size information on your organization. This will help us map processes based on the scale of an organization (rough estimates are fine): Number of employees/users
Number of managed desktops
Number of data centers
How many IT staff
Number of IT staff dedicated to patching

2. Additional organization information:

Additional organization information: Vertical industry
Your job title role
Your role in patch management
Please list any regulations that affect (or require) your patching process

3. If you were provided with a Registration Code, please enter it here:

If you were provided with a Registration Code, please enter it here:

4. My organization's patch management process can be generally characterized as:

My organization's patch management process can be generally characterized as: Do not know

Broadly Mature, with well defined policies, workflow, and tools/resources across the entire organization.

Focused Mature, with well defined policies, workflow, and tools/resources, but only for certain kinds of software/systems (e.g. desktops).

Policy Driven: relies on adherence to policies, lacking strict workflow or tools/resources.

Tool Driven: relies strongly on tools and their workflow, but without defined policies.

Informal: Lacking both policies and tools; an ad-hoc/as needed excercise.

Please add comments, details, or other approaches (optional)

5. Using the following criteria, categorize the maturity of your patch management process for the specified areas:

	Do not know	Mature: well defined policies, workflow, and tools/resources.	Policy Driven: relies on adherence to policies, lacking strict workflow or tools/resources.	Tool Driven: relies strongly on tools and their workflow, but without defined policies.	Informal: Lacking both policies and tools; an ad-hoc/as needed exercise.	N/A: No formal or informal patching.
Workstation OS	*Using the following criteria, categorize the maturity of your patch management process for the specified areas: Workstation OS Do not know	Workstation OS Mature: well defined policies, workflow, and tools/resources.	Workstation OS Policy Driven: relies on adherence to policies, lacking strict workflow or tools/resources.	Workstation OS Tool Driven: relies strongly on tools and their workflow, but without defined policies.	Workstation OS Informal: Lacking both policies and tools; an ad-hoc/as needed exercise.	Workstation OS N/A: No formal or informal patching.
Workstation Applications	Workstation Applications Do not know	Workstation Applications Mature: well defined policies, workflow, and tools/resources.	Workstation Applications Policy Driven: relies on adherence to policies, lacking strict workflow or tools/resources.	Workstation Applications Tool Driven: relies strongly on tools and their workflow, but without defined policies.	Workstation Applications Informal: Lacking both policies and tools; an ad-hoc/as needed exercise.	Workstation Applications N/A: No formal or informal patching.
Workstation Device Drivers and Firmware	Workstation Device Drivers and Firmware Do not know	Workstation Device Drivers and Firmware Mature: well defined policies, workflow, and tools/resources.	Workstation Device Drivers and Firmware Policy Driven: relies on adherence to policies, lacking strict workflow or tools/resources.	Workstation Device Drivers and Firmware Tool Driven: relies strongly on tools and their workflow, but without defined policies.	Workstation Device Drivers and Firmware Informal: Lacking both policies and tools; an ad-hoc/as needed exercise.	Workstation Device Drivers and Firmware N/A: No formal or informal patching.
General Use Server OS	General Use Server OS Do not know	General Use Server OS Mature: well defined policies, workflow, and tools/resources.	General Use Server OS Policy Driven: relies on adherence to policies, lacking strict workflow or tools/resources.	General Use Server OS Tool Driven: relies strongly on tools and their workflow, but without defined policies.	General Use Server OS Informal: Lacking both policies and tools; an ad-hoc/as needed exercise.	General Use Server OS N/A: No formal or informal patching.
General Use Server Applications	General Use Server Applications Do not know	General Use Server Applications Mature: well defined policies, workflow, and tools/resources.	General Use Server Applications Policy Driven: relies on adherence to policies, lacking strict workflow or tools/resources.	General Use Server Applications Tool Driven: relies strongly on tools and their workflow, but without defined policies.	General Use Server Applications Informal: Lacking both policies and tools; an ad-hoc/as needed exercise.	General Use Server Applications N/A: No formal or informal patching.
General Use Server Device Drivers and Firmware	General Use Server Device Drivers and Firmware Do not know	General Use Server Device Drivers and Firmware Mature: well defined policies, workflow, and tools/resources.	General Use Server Device Drivers and Firmware Policy Driven: relies on adherence to policies, lacking strict workflow or tools/resources.	General Use Server Device Drivers and Firmware Tool Driven: relies strongly on tools and their workflow, but without defined policies.	General Use Server Device Drivers and Firmware Informal: Lacking both policies and tools; an ad-hoc/as needed exercise.	General Use Server Device Drivers and Firmware N/A: No formal or informal patching.
Database Management Systems	Database Management Systems Do not know	Database Management Systems Mature: well defined policies, workflow, and tools/resources.	Database Management Systems Policy Driven: relies on adherence to policies, lacking strict workflow or tools/resources.	Database Management Systems Tool Driven: relies strongly on tools and their workflow, but without defined policies.	Database Management Systems Informal: Lacking both policies and tools; an ad-hoc/as needed exercise.	Database Management Systems N/A: No formal or informal patching.
Enterprise Application Servers (e.g. ERP, CRM, document management, other business applications)	Enterprise Application Servers (e.g. ERP, CRM, document management, other business applications) Do not know	Enterprise Application Servers (e.g. ERP, CRM, document management, other business applications) Mature: well defined policies, workflow, and tools/resources.	Enterprise Application Servers (e.g. ERP, CRM, document management, other business applications) Policy Driven: relies on adherence to policies, lacking strict workflow or tools/resources.	Enterprise Application Servers (e.g. ERP, CRM, document management, other business applications) Tool Driven: relies strongly on tools and their workflow, but without defined policies.	Enterprise Application Servers (e.g. ERP, CRM, document management, other business applications) Informal: Lacking both policies and tools; an ad-hoc/as needed exercise.	Enterprise Application Servers (e.g. ERP, CRM, document management, other business applications) N/A: No formal or informal patching.
Web Application Servers	Web Application Servers Do not know	Web Application Servers Mature: well defined policies, workflow, and tools/resources.	Web Application Servers Policy Driven: relies on adherence to policies, lacking strict workflow or tools/resources.	Web Application Servers Tool Driven: relies strongly on tools and their workflow, but without defined policies.	Web Application Servers Informal: Lacking both policies and tools; an ad-hoc/as needed exercise.	Web Application Servers N/A: No formal or informal patching.
Networking Hardware/Software (routers, switches, DHCP, DNS)	Networking Hardware/Software (routers, switches, DHCP, DNS) Do not know	Networking Hardware/Software (routers, switches, DHCP, DNS) Mature: well defined policies, workflow, and tools/resources.	Networking Hardware/Software (routers, switches, DHCP, DNS) Policy Driven: relies on adherence to policies, lacking strict workflow or tools/resources.	Networking Hardware/Software (routers, switches, DHCP, DNS) Tool Driven: relies strongly on tools and their workflow, but without defined policies.	Networking Hardware/Software (routers, switches, DHCP, DNS) Informal: Lacking both policies and tools; an ad-hoc/as needed exercise.	Networking Hardware/Software (routers, switches, DHCP, DNS) N/A: No formal or informal patching.
Infrastructure Applications (directories, security hardware/software, etc.)	Infrastructure Applications (directories, security hardware/software, etc.) Do not know	Infrastructure Applications (directories, security hardware/software, etc.) Mature: well defined policies, workflow, and tools/resources.	Infrastructure Applications (directories, security hardware/software, etc.) Policy Driven: relies on adherence to policies, lacking strict workflow or tools/resources.	Infrastructure Applications (directories, security hardware/software, etc.) Tool Driven: relies strongly on tools and their workflow, but without defined policies.	Infrastructure Applications (directories, security hardware/software, etc.) Informal: Lacking both policies and tools; an ad-hoc/as needed exercise.	Infrastructure Applications (directories, security hardware/software, etc.) N/A: No formal or informal patching.

Comments/Details (Optional)

6. Do you currently collect metrics for:

	Yes	No	Do not know
Patch management effectiveness- how well the organization adheres to policies (e.g. % of systems in compliance or a similar metric):	*Do you currently collect metrics for: Patch management effectiveness- how well the organization adheres to policies (e.g. % of systems in compliance or a similar metric): Yes	Patch management effectiveness- how well the organization adheres to policies (e.g. % of systems in compliance or a similar metric): No	Patch management effectiveness- how well the organization adheres to policies (e.g. % of systems in compliance or a similar metric): Do not know
Patch management efficiency- how well the organization deploys patches (e.g. time to patch):	Patch management efficiency- how well the organization deploys patches (e.g. time to patch): Yes	Patch management efficiency- how well the organization deploys patches (e.g. time to patch): No	Patch management efficiency- how well the organization deploys patches (e.g. time to patch): Do not know

Please list what metrics you use

7. We outsource patch management for the following:

We outsource patch management for the following: Desktops/laptops (operating system) Desktops/laptops (other software) Desktops/laptops (device drivers)

Servers (host operating system)

Database Servers

Enterprise Application Servers (e.g. ERP, CRM, document management, other business applications)

Web Application Servers

Networking Hardware/Software (routers, switches, DHCP, DNS)

Infrastructure Servers (directories, security hardware/software, etc.)

Only some specific applications, based on support contracts

Do not know

Comments/Details (Optional)

Below is a sample patch management process for the next question

8. The following is a list of potential steps in a patch management process (see the diagram above). Please let us know which steps you use in your process, and in the comments field you can describe your process if it's different, or any other steps you use. Since your goal is to understand real-world processes, please only check steps you know match, and use the comments for where you are different.

The following is a list of potential steps in a patch management process (see the diagram above). Please let us know which steps you use in your process, and in the comments field you can describe your process if it's different, or any other steps you use. Since your goal is to understand real-world processes, please only check steps you know match, and use the comments for where you are different. Monitor for Release/Advisory: Anything associated with tracking patch releases, since all vendors follow different processes.

Evaluate: Initial evaluation of the patch. What’s it for? Is it security-sensitive? Do we use that software? Is the issue relevant in our environment? Are there workarounds or dependencies?

Acquire: Get the patch.

Prioritize and Schedule: Prioritize based on the nature of the patch itself, and your infrastructure/assets. Then build out a deployment schedule, based on your prioritization.

Test and Approve: Perform any required testing of the raw patch, and approve the patch for release.

Create and Test Deployment Package: Prepare the patch for deployment, and test the deployment package, scripts, etc..

Deploy

Confirm Deployment: Verify that patches were properly deployed. This might include use of configuration management or vulnerability assessment tools.

Clean up: Clean up any bad deployments, remnants of the patch application procedure, or other associated cruft/detritus.

Document and Update Configuration Standards: Document the patch deployment, which may be required for regulatory compliance, and update any associated configuration standards/guidelines/requirements.

Other (please specify)

9. If you are willing to be interviewed to determine the resources you dedicate for patch management, please give us your email address for follow up:

If you are willing to be interviewed to determine the resources you dedicate for patch management, please give us your email address for follow up:

10. We use the following sources and methods to know when patches are released:

We use the following sources and methods to know when patches are released: Vendor email lists

Vendor blogs

Internal software notifications (e.g. a version check popup window)

Third party email lists (paid subscription)

Third party email lists (free)

Third party services (managed service)

Third party tool (e.g. a patch management tool with an internal update feed)

Media/news

CVE/NVD

US-CERT Advisories

Information Assurance Vulnerability Messages

Information Security Vulnerability Messages

None

Do not know

Other (please specify)

11. When a patch is released, the following roles/teams are involved in evaluating the patch for possible deployment:

When a patch is released, the following roles/teams are involved in evaluating the patch for possible deployment: Security

Network operations

Workstation/systems administration

Application owners (including DBAs)

General IT management (e.g. CIO)

All/some of the above, depending on the involved software

Do not know

Other (please specify)

12. Please rank how the following factors help determine a patch priority (rank in order, with 1 being the most important):

Please rank how the following factors help determine a patch priority (rank in order, with 1 being the most important): Value of the asset being patched (e.g. a financial system vs. low-value application)
The criticality/importance of the patch (stability, functionality)
The security priority/importance of the patch
Resources required to roll out the patch
Potential costs (resources/time)
Business unit requests
Business unit resistance

13. How are patches and/or deployment packages tested before deployment?

How are patches and/or deployment packages tested before deployment? Regression testing

Stability testing

Interoperability testing

No formal testing

Testing varies by asset

Other (please specify)

14. Please describe which kinds of tools you use for which kinds of systems:

	Third party, cross platform patch management tool	Third party, single platform patch management tool	Vendor/product feature of our software (e.g. built-in autoupdate mechanism)	Vendor/product patch management tool for specific products (e.g. an external patch management tool for the product by the same vendor)
Workstation OS	*Please describe which kinds of tools you use for which kinds of systems: Workstation OS Third party, cross platform patch management tool	Workstation OS Third party, single platform patch management tool	Workstation OS Vendor/product feature of our software (e.g. built-in autoupdate mechanism)	Workstation OS Vendor/product patch management tool for specific products (e.g. an external patch management tool for the product by the same vendor)
Workstation Applications	Workstation Applications Third party, cross platform patch management tool	Workstation Applications Third party, single platform patch management tool	Workstation Applications Vendor/product feature of our software (e.g. built-in autoupdate mechanism)	Workstation Applications Vendor/product patch management tool for specific products (e.g. an external patch management tool for the product by the same vendor)
Workstation Device Drivers and Firmware	Workstation Device Drivers and Firmware Third party, cross platform patch management tool	Workstation Device Drivers and Firmware Third party, single platform patch management tool	Workstation Device Drivers and Firmware Vendor/product feature of our software (e.g. built-in autoupdate mechanism)	Workstation Device Drivers and Firmware Vendor/product patch management tool for specific products (e.g. an external patch management tool for the product by the same vendor)
General Use Server OS	General Use Server OS Third party, cross platform patch management tool	General Use Server OS Third party, single platform patch management tool	General Use Server OS Vendor/product feature of our software (e.g. built-in autoupdate mechanism)	General Use Server OS Vendor/product patch management tool for specific products (e.g. an external patch management tool for the product by the same vendor)
General Use Server Applications	General Use Server Applications Third party, cross platform patch management tool	General Use Server Applications Third party, single platform patch management tool	General Use Server Applications Vendor/product feature of our software (e.g. built-in autoupdate mechanism)	General Use Server Applications Vendor/product patch management tool for specific products (e.g. an external patch management tool for the product by the same vendor)
General Use Server Device Drivers and Firmware	General Use Server Device Drivers and Firmware Third party, cross platform patch management tool	General Use Server Device Drivers and Firmware Third party, single platform patch management tool	General Use Server Device Drivers and Firmware Vendor/product feature of our software (e.g. built-in autoupdate mechanism)	General Use Server Device Drivers and Firmware Vendor/product patch management tool for specific products (e.g. an external patch management tool for the product by the same vendor)
Database Management Systems	Database Management Systems Third party, cross platform patch management tool	Database Management Systems Third party, single platform patch management tool	Database Management Systems Vendor/product feature of our software (e.g. built-in autoupdate mechanism)	Database Management Systems Vendor/product patch management tool for specific products (e.g. an external patch management tool for the product by the same vendor)
Enterprise Application Servers (e.g. ERP, CRM, document management, other business applications)	Enterprise Application Servers (e.g. ERP, CRM, document management, other business applications) Third party, cross platform patch management tool	Enterprise Application Servers (e.g. ERP, CRM, document management, other business applications) Third party, single platform patch management tool	Enterprise Application Servers (e.g. ERP, CRM, document management, other business applications) Vendor/product feature of our software (e.g. built-in autoupdate mechanism)	Enterprise Application Servers (e.g. ERP, CRM, document management, other business applications) Vendor/product patch management tool for specific products (e.g. an external patch management tool for the product by the same vendor)
Web Application Servers	Web Application Servers Third party, cross platform patch management tool	Web Application Servers Third party, single platform patch management tool	Web Application Servers Vendor/product feature of our software (e.g. built-in autoupdate mechanism)	Web Application Servers Vendor/product patch management tool for specific products (e.g. an external patch management tool for the product by the same vendor)
Networking Hardware/Software (routers, switches, DHCP, DNS)	Networking Hardware/Software (routers, switches, DHCP, DNS) Third party, cross platform patch management tool	Networking Hardware/Software (routers, switches, DHCP, DNS) Third party, single platform patch management tool	Networking Hardware/Software (routers, switches, DHCP, DNS) Vendor/product feature of our software (e.g. built-in autoupdate mechanism)	Networking Hardware/Software (routers, switches, DHCP, DNS) Vendor/product patch management tool for specific products (e.g. an external patch management tool for the product by the same vendor)
Infrastructure Applications (directories, security hardware/software, etc.)	Infrastructure Applications (directories, security hardware/software, etc.) Third party, cross platform patch management tool	Infrastructure Applications (directories, security hardware/software, etc.) Third party, single platform patch management tool	Infrastructure Applications (directories, security hardware/software, etc.) Vendor/product feature of our software (e.g. built-in autoupdate mechanism)	Infrastructure Applications (directories, security hardware/software, etc.) Vendor/product patch management tool for specific products (e.g. an external patch management tool for the product by the same vendor)

Please list any specific tools you use

15. Our patch management tool(s) cover the following platforms:

Our patch management tool(s) cover the following platforms: Desktops/laptops (operating system)

Desktops/laptops (other software)

Desktops/laptops (device drivers)

Servers (host operating system)

Database Servers

Enterprise Application Servers (e.g. ERP, CRM, document management, other business applications)

Web Application Servers

Networking Hardware/Software (routers, switches, DHCP, DNS)

Infrastructure Servers (directories, security hardware/software, etc.)

Our tools work with specific products, not general categories/platforms

Comment/Details (Optional)

16. We validate successful patch deployment via:

We validate successful patch deployment via: Reports from a patch management system

Automated testing using an external tool

Manual testing

Internal system/application reporting/feature

Vulnerability scanning

Configuration scanning

User complaints

Other/Comments

17. How often do you validate registration of IT assets in the patch management tool? (Number of days, or fraction thereof)

How often do you validate registration of IT assets in the patch management tool? (Number of days, or fraction thereof)

18. We formally document configuration changes

We formally document configuration changes Always

Often

Sometimes

Never

Comments/Details

19. Are patches managed by inclusion or exclusion?

Are patches managed by inclusion or exclusion? Inclusion- everything is patched by default, and exceptions explained

Exclusion- patches applied when directed only

Comments/Details (Optional)

20. Do you:

	Yes	No
Have a formal change control process that authorizes patches to be released and schedules the release?	*Do you: Have a formal change control process that authorizes patches to be released and schedules the release? Yes	Have a formal change control process that authorizes patches to be released and schedules the release? No
Have a defined, recurring "patch window"?	Have a defined, recurring "patch window"? Yes	Have a defined, recurring "patch window"? No
Have a defined, recurring "reboot window"?	Have a defined, recurring "reboot window"? Yes	Have a defined, recurring "reboot window"? No
Patch during business hours in general?	Patch during business hours in general? Yes	Patch during business hours in general? No
Patch during business hours based on technology type?	Patch during business hours based on technology type? Yes	Patch during business hours based on technology type? No
Force reboots after patching or...?	Force reboots after patching or...? Yes	Force reboots after patching or...? No
... allow administrators or users to reboot on their own?	... allow administrators or users to reboot on their own? Yes	... allow administrators or users to reboot on their own? No
Vary reboot requirements by technology type?	Vary reboot requirements by technology type? Yes	Vary reboot requirements by technology type? No
Report patch compliance as a metric?	Report patch compliance as a metric? Yes	Report patch compliance as a metric? No

Comments/Details (Optional)

21. If you would like us to contact you for a more-detailed interview, please provide your email address:

If you would like us to contact you for a more-detailed interview, please provide your email address: