As a result, IT leaders in smaller organisations are putting growth and competitiveness at risk by believing they don't need an enterprise architecture, the analyst firm said in a recent research note.
"Clearly small to medium businesses are not going to build a fully-fledged enterprise architecture because they simply don't have the resources to do that," Gartner research VP Brian Burke said.
"But what we find is they often under-invest as well so they're not really developing".
That under-investment may be down to a mistaken belief among some smaller companies that an enterprise architecture can't achieve the same business value and impact for them as it might for a larger firm.
However, Gartner's research suggests that the opposite is true because smaller businesses have several key advantages over larger organisations when it comes to pursuing a business-driven approach to technology.
For example, smaller firms will be at an earlier stage of creating an enterprise architecture so they will not be limited by legacy, past failures or obsolete approaches that might otherwise have become engrained in the organisation.
Misconceptions about the place of an enterprise architecture in smaller companies may also be down to much of the literature and research on the subject.
"Most of what you read about enterprise architecture speaks to larger organisations and makes assumptions about the resources that they're able to commit," Burke said.
"So small businesses need to be both realistic and selective about where they are going to focus their energy, because even large organisations struggle with building a full-out enterprise architecture."
Burke said maturity in this area across organisations in general is relatively low. The research found that 41 percent of the smallest organisations are still at Gartner's lowest maturity ranking for enterprise architecture, compared with 30 percent of high-end mid-size firms.
"What we find is most organisations are still struggling with enterprise architecture but more so within small and medium businesses," he said.
Many are struggling because they are giving it little thought thanks to constant pressures on business and IT resources.
"Often in IT where enterprise architecture is going to be an issue, there are always projects that need to be completed, and small to medium businesses tend not to be able to, or choose not to, allocate resources to these higher level disciplines," he said.
The result is a piecemeal approach whereas in the longer term a strategic initiative would be more beneficial.
"That's the challenge. Change is driven by projects and the fastest way to the end point of a project is to isolate it," Burke said.
"So decisions are made isolated within projects and that results in stovepipes of information and applications and technologies that ultimately have to be integrated across the business. That increases the level of complexity over time."
That approach contrasts with having some oversight that is focused on optimising activities at the enterprise level, which can avoid some of those integration challenges and complexity issues over time.
"As you make poor decisions, either within projects or across the organisation, you build up this architectural debt where those poor decisions that you made accumulate. There is a real cost to that architectural debt as well," Burke said.
"So while you might speed through a project and get the project delivered as efficiently and as effectively as possible, depending on how you made those decisions along the way, you may have built up or contributed to the architectural debt for the solution that's delivered and the organisation overall."
The importance of enterprise architecture lies in raising the level of decision making.
"It ensures that decision making is done at the appropriate level. When decision making is done on a project, it's often by the people writing the code, the developers themselves. They can make all kinds of decisions. So the overall risk is that inappropriate decisions are going to be made," Burke said.
"The kinds of decisions that will be determined to be inappropriate could be decisions as broad as, 'We've started working on projects that are high risk low value' - those kinds of things, right down to, 'We've made bad technology or design choices that are going to limit our functionality or increase our costs over time'."
So enterprise architecture is about planning these approaches and ensuring decisions are made at an appropriate level.
"It doesn't mean that we're going to make all the decisions for a project but what it does mean is that we're going to provide direction on those major decisions and hopefully avoid some of those major bad decisions that projects often make," Burke said.
Appointing someone to take charge of developing an enterprise architecture for a smaller business is an important first step, he said. That role can be shared or part of wider responsibilities. Creating an appropriate scope for the architecture that integrates business and IT strategies is also vital.
Gartner also suggests thinking in terms of delivering business value from the enterprise architecture in weeks and months, rather than years.
Developing metrics and a clear way of expressing the architecture's value to the business is another recommendation.