ILTA Artificial Intelligence and Machine Learning Survey

ILTA's 2019 Artificial Intelligence and Machine Learning Survey Mission Statement:

Almost a third of responses in ILTA’s 2018 Technology Survey cited “Artificial Intelligence and Machine Learning” as a technology or trend they believe will create significant change or be a major factor in the legal technology profession. 100% of the largest firms (over 700 attorneys) indicated they are pursuing AI/ML projects, and more than a third said they had “one or more AI/ML tools in production.” There is growing recognition that these tools/disciplines will play a significant part in legal strategy across the industry.

The ILTA Artificial Intelligence & Machine Learning Survey will take a deep dive into the “AI/ML” landscape in the legal industry, providing insights into the business problems being solved by these tools, the ways the tools are being used, how they are being deployed and trained, and just how widely they are being adopted by the legal profession.

Question Title

* 1. How long do you believe it will be before AI-powered software will create fundamental change in the legal industry?

Already There

1 - 3 Years

4 - 6 Years

7 - 9 Years

10+ Years

Question Title

* 2. What three legal business uses do you believe AI-powered software will attain acceptable maturity in first?


Legal business uses	Legal business uses menu	Legal business uses menu	Legal business uses menu

Question Title

* 3. What % of work done by attorneys (both legal-specific and nonlegal work done by attorneys) do you believe will be replaced by AI-powered software in the next 5 years?

Question Title

* 4. List up to three legal AI Software in use or POC in your organization:

	Choice #1	Choice #2	Choice #3
List of Tools	List of Tools Choice #1 menu	List of Tools Choice #2 menu	List of Tools Choice #3 menu

Question Title

* 5. Does your organization have a strategic plan to incorporate AI-powered software into business operations?

Yes, We are past strategic, with operational plans in place

Yes, integral to strategic plan

Yes, but not called out specifically

Notionally

Not yet

No & we don't need to

Question Title

* 6. Does your organization have a specific budget line for purchase or development of AI-powered software?

Yes

Yes - Combined with one or two other Items

Plan is to include a budget line within the next 12 months

Question Title

* 7. What are the elements of the business case/ROI/budget for purchase or development of AI-powered software?

R&D

Innovation

Asset of the organization

Pass through cost to client/customer

Internal expense for cost management and futuristic savings to be passed on internally or to a client

Marketing, perception or keeping up with the Jones"

Question Title

* 8. Who in your organization leads legal AI initiatives?


Please pick more than one if they apply:	Please pick more than one if they apply: menu	Please pick more than one if they apply: menu	Please pick more than one if they apply: menu

Question Title

* 9. What is the biggest challenge in advancing adoption of AI in your organization?

Cost of Product

Integrations

Good Data Sets

Skilled resources

Training the tool

Change Management

Understanding the Tools

Understanding the Need for the Tools

Creating the Urgency and Need Internally

Question Title

* 10. Rank the relevancy or importance of these categories of AI-powered software to your organization:

	0%	1 - 24%	25 - 49%	50 - 74%	75 - 100%
Billing/Spend Management	Billing/Spend Management 0%	Billing/Spend Management 1 - 24%	Billing/Spend Management 25 - 49%	Billing/Spend Management 50 - 74%	Billing/Spend Management 75 - 100%
Contract (Post-Execution)	Contract (Post-Execution) 0%	Contract (Post-Execution) 1 - 24%	Contract (Post-Execution) 25 - 49%	Contract (Post-Execution) 50 - 74%	Contract (Post-Execution) 75 - 100%
Contract (Pre-Execution)	Contract (Pre-Execution) 0%	Contract (Pre-Execution) 1 - 24%	Contract (Pre-Execution) 25 - 49%	Contract (Pre-Execution) 50 - 74%	Contract (Pre-Execution) 75 - 100%
E-Discovery/Document Review	E-Discovery/Document Review 0%	E-Discovery/Document Review 1 - 24%	E-Discovery/Document Review 25 - 49%	E-Discovery/Document Review 50 - 74%	E-Discovery/Document Review 75 - 100%
Expertise Automation	Expertise Automation 0%	Expertise Automation 1 - 24%	Expertise Automation 25 - 49%	Expertise Automation 50 - 74%	Expertise Automation 75 - 100%
Insight/Predictive Tools	Insight/Predictive Tools 0%	Insight/Predictive Tools 1 - 24%	Insight/Predictive Tools 25 - 49%	Insight/Predictive Tools 50 - 74%	Insight/Predictive Tools 75 - 100%
Legal Research	Legal Research 0%	Legal Research 1 - 24%	Legal Research 25 - 49%	Legal Research 50 - 74%	Legal Research 75 - 100%
Risk Assessment	Risk Assessment 0%	Risk Assessment 1 - 24%	Risk Assessment 25 - 49%	Risk Assessment 50 - 74%	Risk Assessment 75 - 100%
Security	Security 0%	Security 1 - 24%	Security 25 - 49%	Security 50 - 74%	Security 75 - 100%

Question Title

* 11. Which practice area(s) or internal department(s) are involved with/responsible for AI tools?


Please pick all that apply	Please pick all that apply menu	Please pick all that apply menu	Please pick all that apply menu

Question Title

* 12. How likely is your organization to develop or purchase a legal AI-powered software to solve an internal business challenge?

One or more tools in production

POC

Planning

Ideation

Not Contemplated

Question Title

* 13. How likely is your organization to develop or purchase a legal AI-powered software to solve a business challenge for clients, outside counsel or customers?

One or more tools in production

POC

Planning

Ideation

Not Contemplated

Question Title

* 14. Are you developing or purchasing the tool?

Purchasing Out-of-the-box tool

Customizing an Out-of-the-box tool

Purchasing a platform and developing a tool within that platform

Developing a tool from scratch

Question Title

* 15. Where will the data and legal AI tool reside?

On Premises

Saas

Cloud-hosted

Question Title

* 16. If you are developing/planning to develop your own AI-powered software or platform, what are you using/considering using to build your models?

Open Source

Azure Cognitive Services

IBM Watson

Other

Question Title

* 17. How broad is the use of legal AI-powered software by your organization?

TBD

Single Department/Practice

Multiple Departments/Practices

Regional

Americas

EMEA

Asia

Global/Organization-wide

Question Title

* 18. What roles are/will be involved in any AI-powered software purchase, development, training and deployment?


Please pick more than one if they apply:	Please pick more than one if they apply: menu	Please pick more than one if they apply: menu	Please pick more than one if they apply: menu

Other (please specify)

Question Title

* 19. What do you believe is the biggest challenge with the available current Data Set for the AI-powered software(s) you are developing or purchasing?

Data not cleansed or normalized

Insufficient data

Incorrect data

Biased data

Data not available

No data challenges

Question Title

* 20. What do you believe is the biggest challenge with training AI-powered software?

Inability to get good training data

Unavailability of skilled personnel to do the training

Insufficient knowledge of data for training

Unpredictable results

Analysis Time for training and validation of results

Data viability for usage

Biased training methods competing for attention

Question Title

* 21. What three elements of quality assurance do you incorporate in your AI projects?

Documented Requirements

Validation Documentation

Documented Success Metrics

Full POC

Run Dual projects (Manual alongside AI)

Question Title

* 22. Is it important for law firms and law departments to lead the adoption of AI-powered software through collboration fellowship or through a exclusivity agreements by one party leading the other?

Yes - Mutual collaboration between law firm and law departments

Adoption by law firms first followed by law departments

Law Department adoption first requesting law firm acceptance

ILTA Artificial Intelligence and Machine Learning Survey

ILTA's 2019 Artificial Intelligence and Machine Learning Survey Mission Statement:

Question Title

* 1. How long do you believe it will be before AI-powered software will create fundamental change in the legal industry?

Question Title

* 2. What three legal business uses do you believe AI-powered software will attain acceptable maturity in first?

Question Title

* 3. What % of work done by attorneys (both legal-specific and nonlegal work done by attorneys) do you believe will be replaced by AI-powered software in the next 5 years?

Question Title

* 4. List up to three legal AI Software in use or POC in your organization:

Question Title

* 5. Does your organization have a strategic plan to incorporate AI-powered software into business operations?

Question Title

* 6. Does your organization have a specific budget line for purchase or development of AI-powered software?

Question Title

* 7. What are the elements of the business case/ROI/budget for purchase or development of AI-powered software?

Question Title

* 8. Who in your organization leads legal AI initiatives?

Question Title

* 9. What is the biggest challenge in advancing adoption of AI in your organization?

Question Title

* 10. Rank the relevancy or importance of these categories of AI-powered software to your organization:

Question Title

* 11. Which practice area(s) or internal department(s) are involved with/responsible for AI tools?

Question Title

* 12. How likely is your organization to develop or purchase a legal AI-powered software to solve an internal business challenge?

Question Title

* 13. How likely is your organization to develop or purchase a legal AI-powered software to solve a business challenge for clients, outside counsel or customers?

Question Title

* 14. Are you developing or purchasing the tool?

Question Title

* 15. Where will the data and legal AI tool reside?

Question Title

* 16. If you are developing/planning to develop your own AI-powered software or platform, what are you using/considering using to build your models?

Question Title

* 17. How broad is the use of legal AI-powered software by your organization?

Question Title

* 18. What roles are/will be involved in any AI-powered software purchase, development, training and deployment?

Question Title

* 19. What do you believe is the biggest challenge with the available current Data Set for the AI-powered software(s) you are developing or purchasing?

Question Title

* 20. What do you believe is the biggest challenge with training AI-powered software?

Question Title

* 21. What three elements of quality assurance do you incorporate in your AI projects?

Question Title

* 22. Is it important for law firms and law departments to lead the adoption of AI-powered software through collboration fellowship or through a exclusivity agreements by one party leading the other?

Question Title

* 23. What steps are you taking to manage and/or eliminate bias in your organization's AI-powered software?

Question Title

* 24. Additional Comments regarding AI/ML/NLP. Is there a question you would like to see added to the next version of this survey?