RSM Middle Market AI Survey 2024: U.S. and Canada

Key takeaways

1. More than three-quarters of middle market companies are currently using AI and generative AI.
2. More than half of companies found generative AI harder to implement than expected.
3. Two-thirds of respondents felt outside help is necessary to optimize generative AI solutions.

Chatbots. Large language models. Predictive analytics. These and other fast-evolving artificial intelligence (AI) tools are rapidly transforming the way businesses operate. A large majority of middle market organizations are embracing AI, lauding its positive impact and boosting resources around it, research conducted on behalf of RSM shows. At the same time, many of these companies are only in the early stages of AI adoption, wrestling with implementation and aware they need outside help to position the technology for the greatest strategic benefit to their businesses while also mitigating risk.

Of some 500 executives polled at midsize organizations in the United States and Canada, more than three-quarters (78%) said AI is being used across their operations, either formally or informally. Seventy-seven percent said their organizations have adopted generative AI solutions such as ChatGPT and Microsoft Copilot. Primary use cases for generative AI are broad, ranging from data analytics to customer service.

With the technological advances and pressure to innovate, many companies understandably want to move fast with AI technology. However, a practical approach is often better as companies understand how best to use the technology and manage its deployment.

A plurality (41%) of respondents in the survey, conducted from Feb. 26 to March 4, 2024, by Big Village Insights said their businesses have only partially implemented AI technology. Just one-fifth had achieved full integration.

“Middle market companies are rapidly understanding AI’s potential,” said RSM US Principal George Casey. “But many are also finding out that implementation is not always simple.

With the technology’s fast pace and complexity, many companies do not have experienced internal resources, and external support and advice are necessary, especially as AI-driven processes become more commonplace and critical for growth.”

Despite the nascent stages of AI adoption for most, the use of the technology has been surprisingly positive; in fact, 85% of respondents using generative AI indicated its impact has exceeded expectations, with benefits including increased efficiency, better customer service and lower costs. Even so, more than half (54%) of these respondents said the technologies have also been more difficult to roll out than expected, and 67% acknowledged that outside help is needed to maximize the benefit of their chosen generative AI solutions.

The survey, which targeted respondents at businesses with annual revenues of $10 million to $1 billion, indicates that nearly three-quarters of those using generative AI (74%) were focusing their dollars squarely on generative AI investments. Among this group, nine in 10 plan to boost their budgets around AI technology and implementation for the next fiscal year.

“As use cases for AI technology expand, companies will need to make a bigger financial and resource commitment to strengthen their overall strategy,” says Casey. “Successfully implementing AI will be an ongoing journey, but it is an exciting time for companies to discover where it can make the most impact on operations.”

Key takeaways

1. Slightly more companies are using AI and generative AI in the U.S. than in Canada.
2. U.S. companies appear to be more mature with AI implementations than their Canadian counterparts.
3. Most companies in the U.S. and Canada expect generative AI budgets to increase next year.

The research in the RSM survey encompassed AI data from middle market executives in the United States (407 respondents) and Canada (103 respondents). While the opinions and strategies of the two countries were generally similar, the data demonstrated some differences in generative AI deployment and goals.

In the survey, slightly more U.S. middle market executives reported using AI tools either formally or informally (80%) than their Canadian counterparts (73%). The data is similar for generative AI, with 78% of U.S. organizations using the technology in business practices, compared to 74% of Canadian respondents.     

With the technological advances and pressure to innovate, many companies understandably want to move fast with AI technology. However, a practical approach is often better as companies understand how best to use the technology and manage its deployment.

A plurality (41%) of respondents in the survey, conducted from Feb. 26 to March 4, 2024, by Big Village Insights said their businesses have only partially implemented AI technology. Just one-fifth had achieved full integration.

However, companies in the U.S. appear to have more mature generative AI strategies than Canadian peers. In the survey, 61% of U.S. companies say they have full or partial integration of generative AI technology, while 37% of Canadian organizations have reached that level.

The main goal for adopting generative AI differs for executive groups in the two countries. U.S. respondents focused on improving quality control (62%), and Canadian respondents set their focus on automating repetitive tasks (61%). However, the next two leading goals for the technology were the same in each country: enhancing customer service (both 51%) and increasing employee productivity and creativity (44% in the U.S. versus 46% in Canada).

A main difference between the two countries is how they will handle budgets for generative AI in the next fiscal year. Among users who have a budget dedicated to generative AI investments, 92% of U.S. middle market executives expect an increase in their generative AI budget, while 78% of Canadian respondents expect theirs to rise.

“Companies in the United States are increasing their budgets for generative AI solutions at a quicker pace than Canadian organizations, which influences the speed of implementation and integration,” said RSM Canada Partner Rhys Morgan. “But the majority of companies in both countries appear poised to continue making significant operational improvements with emerging generative AI innovations.” 

Key takeaways

1. The majority of middle market companies have begun implementing AI and generative AI solutions. 
2. Most organizations report being in the early stages of AI adoption.
3. The most popular generative AI tools include ChatGPT, Microsoft Copilot and Adobe/Firefly.

AI has seen a meteoric rise in the middle market during the last year. Technology recently seen as unrealistic or out of reach for many companies, has rapidly become essential to optimize efficiency, improve the internal and external user experience and generate timely insights to drive informed decision making. 

RSM research shows that the majority of middle market companies have already begun integrating AI initiatives into their businesses. In fact, 78% of survey respondents say their organization either formally or informally uses AI, and 77% report using generative AI. For companies that do not currently use AI, nearly half (46%) have a plan to use AI in the next 12 months.

However, while many middle market organizations already use AI, significant opportunities remain. Organizations tend to be in the early stages of adoption, as most executives in RSM’s survey (41%) report only partially implementing traditional AI solutions, such as rules-based applications, to detect patterns and make decisions. Executives that use generative AI report similar progress, with 34% of respondents indicating partial implementation of the advanced self-sufficient content and data creation platforms. 

“Generally, middle market companies have only begun to dip their toes in the water of AI tools and applications,” says Casey. “With an effective implementation approach, companies will continue to see the power and value of the technology as strategies continue to mature.”

Defining a measured pathway to AI is often the best strategy for middle market organizations. Many midsize companies do not have the budget or personnel flexibility to develop and manage successful broad AI implementations across their entire businesses. Instead, the smarter approach is to focus on AI integration projects within key business functions or units.

“Companies can make a significant difference in their operations by undergoing an analysis and prioritizing where AI can make the most impact,” says RSM US Director Robbie Beyer. “An ‘everything all at once’ approach to AI is often unrealistic, but incremental progress will start to build momentum and inspire adoption and increased confidence throughout the organization.”   

As the popularity of AI increases, organizations now have access to a broader range of tools and applications. RSM data shows that ChatGPT is the most widely used AI technology (76%), followed by chatbots (39%) and Google Cloud AI (34%). Among those using generative AI, the most widely used platforms include ChatGPT (82%), Microsoft Copilot (30%), Adobe/Firefly (24%), Google Gemini (formerly known as Bard)/Duet (22%), Azure AI (21%) and Zoom AI (20%).

Many middle market companies are relatively early in their AI journey and eager to experience change; however, with so many emerging solutions, they must practice discretion when choosing and implementing tools. The right solutions can transform key business processes, but not every platform is a good fit for every company.  

“Every business is different with different needs,” Casey says. “With new solutions regularly emerging and rapidly evolving, companies must make sure they conduct a thorough selection process before implementing AI technology. With that, solutions are much more likely to meet expectations and deliver a return on investment.”

Data quality is another crucial consideration that can be overlooked when evaluating potential AI solutions. Taking full advantage of AI requires a strong data foundation because it determines the reliability of outputs from the technology.

“Quality data is a critical element in any AI project and it should be one of the first considerations for determining the viability of a use case,” says RSM Canada Director Sawan Dhaliwal. “Poorly structured or incomplete data can directly lead to inaccurate results or flawed outcomes and diminish confidence in solutions.”

Key takeaways

1. Nearly three-quarters of middle market companies have a dedicated generative AI budget.
2. Almost nine out of 10 respondents expect to increase their generative AI budget next year.

Traditional AI solutions that perform activities such as analyzing historical data to answer questions or recognizing patterns based on rules remain valuable in the middle market. However, generative AI solutions are now in the spotlight because they take those capabilities one step further by learning from large data sets to create new data and information and simplifying tasks such as writing, editing, research, and video and audio development.

Generative AI represents the next generation of AI technology, with exciting advances in creativity and functionality, but it is also more complex and requires a greater financial commitment.

RSM’s data shows that the majority of middle market companies understand the funding and structure necessary to drive the implementation of generative AI innovations. Seventy-four percent of respondents using generative AI report their organization has a budget dedicated to generative AI investments.  

“Investing in generative AI is more than a one-time endeavor, as the funding must be available as technology continues to advance and new use cases are discovered,” says RSM US Principal Diego Rosenfeld. “As time goes by, the budget will require careful oversight to take advantage of opportunities to stay ahead of competition.” 

With the momentum behind AI tools and solutions, nearly all middle market organizations in RSM’s survey that have a dedicated budget for generative AI investments are focused on expanding their current generative AI programs. In fact, 89% of respondents say their organizations plan to increase their generative AI budget in the coming fiscal year—56% plan to increase it somewhat, while 33% are set for a substantial increase.  

“A larger budget does not always mean greater results,” says Casey. “Successful planning and experience are critical to generative AI success, but combining those with financial flexibility greatly increases the probability of creating transformative change within an organization.”

Key takeaways

1. Most companies feel generative AI has had more of a positive impact than anticipated.
2. More than half of midsize companies use generative AI for quality control and customer service.

Companies see few limits to generative AI’s potential. Solutions and applications can solve a host of problems, such as enhancing the customer experience, summarizing large amounts of data, becoming more creative and increasing productivity. New use cases are emerging every day, and middle market companies have a tremendous opportunity to differentiate themselves from the competition with how they leverage generative AI technology.

“Generative AI has the potential to be a game-changer for middle market enterprises," says Beyer. "It grants them opportunities for efficiency and transformation in a scalable and cost-effective manner that was previously unattainable."

Responses in the RSM survey demonstrate the breadth of generative AI solutions used by middle market companies. More than half seek to improve quality control (58%). Other top goals include enhancing customer service (51%), automating repetitive tasks (45%) and increasing employee productivity and creativity (45%). 

Even though many midsize companies are relatively early in their generative AI journey, the technology is already yielding big benefits. RSM survey data shows that generative AI has resulted in more positive change in the middle market than initially anticipated. Eighty-five percent of respondents either somewhat or completely agreed that generative AI has positively influenced their organization more than expected. 

Executives shared several specific generative AI success stories, mainly focusing on increased productivity and efficiency, enhanced customer service and satisfaction, and reduced costs.  

“The majority of organizations have only scratched the surface on the impact generative AI can have within their organizations,” says Rosenfeld. “The technology will be a major factor in guiding how companies manage many key functions in the future.” 

Key takeaways

1. Data and security is a leading concern for companies not using generative AI.
2. Many companies not using generative AI think integration may be difficult.
3. Generative AI can be difficult to implement, emphasizing the importance of outside assistance.

The potential for generative AI to increase efficiency and optimize critical business processes is very real, but implementation does not come without challenges. Ignoring potential risks could derail the progress of AI implementation projects and limit the value of investments.

Security and privacy is a primary concern for all businesses, especially as legislation in the United States, Canada and internationally becomes more restrictive. Among companies in the RSM survey that say they don’t plan on using generative AI in the next 12 months, 46% have concerns over data privacy and security. 

When considering generative AI investments, many companies are concerned that data sent to large language models can lead to vulnerabilities or unintended exposure. However, if the right processes are in place, companies should be able to retain the desired level of control over their data. Issues can occur, but they would likely indicate a more widespread data control issue. 

“Many data security and privacy concerns are not exclusive to AI,” said RSM US Director Dave Mahoney. “Instead, they are often data loss problems that already exist and extend beyond AI applications. Implementing effective data controls is not a new concept, but they now need to be applied to AI solutions.”  

In addition, 44% of respondents who have yet to implement generative AI believe it is too difficult to integrate into existing workflows and operations. 

“Not at least evaluating generative AI’s potential to transform business operations would be a mistake,” said Casey. “Companies of all sizes and in all industries are now leveraging generative AI technology, and those that don’t utilize it in some way risk being left behind.”

While generative AI has had a positive impact overall on the middle market, many companies have experienced difficulty with implementation. In the RSM survey, 54% of executives said it has been harder to implement than expected. Another 67% report they need outside help to get the most out of their generative AI solutions. In the open commentary of the survey, respondents suggested common challenges, such as getting employee buy-in, a lack of internal skills and, again, concerns over data security and privacy. 

With generative AI evolving so quickly, internal personnel at middle market companies often do not have extensive experience with innovations and commonly struggle to execute an effective implementation and integration process. 

“Almost every middle market company requires external guidance to design and implement an effective generative AI approach,” said Casey. “Internal experience and business needs simply do not match very often. Companies understand they need help to determine what solutions are the best fit, align them with company objectives and optimize the return on investment.”    

In addition to external assistance, Mahoney stresses the importance of leveraging a framework for responsible generative AI adoption. “Generative AI works best with a governance plan in place,” he said. “It enables you to unlock potential opportunities and gain insights faster while also actively addressing potential risks.”