Architecture and Digital Platforms in Complex Organisations

“An individual building, the style in which it is going to be designed and built, is not that important. The important thing, really, is the community. How does it affect life?” I.M. Pei

The challenge we face

Building enterprises in the digital age is exciting on three levels.

The opportunity for social and commercial impact has never been greater, and the link to a sense of purpose and meaning has never been more important.

The impatient expectations that our clients and employees have about the outcomes of these endeavors are growing exponentially.

The challenge of executing has never been more complex, while the global community of support and learning has also never been as good.

It is the architecture of our business platforms that will allow us to embrace these opportunities.

How do we do architecture that impacts life?

I use the term “architecture”, to refer to a business platform of which technology is a key enabler, but not the only consideration. This could be referred to as “enterprise architecture”, but I find that term has baggage that I want to avoid.

There are many good books written about the technical disciplines and methodologies of enterprise architecture. Immensely comprehensive frameworks have been established and, in general, there is a high level of alignment in the industry around these. Unfortunately, the span of what needs to be done is so vast that it is not always easy to know where to begin. My hope is that these ideas will help architecture practices to find faster ways to deliver real business impact.

The ideas presented are implicitly aimed at complex, knowledge-based organisations, in which designing strategic alignment and executing on this alignment are complex at both a human and a technical level.

Why do we do architecture?

At its heart, architecture is about strategy execution. Most organisations struggle to translate their business vision into an executable strategy that connects and aligns all of its parts and makes it responsive to client needs in a radical way.

Architecture is also fundamentally about protecting the ability of the organisation to execute with f_low_. This is a topic on its own, but some key elements are:

• when a decision must be made on how to execute, we should have the people and insight to make the decision with confidence and without delays. This takes deliberate and continuous learning and planning;
• we need a bias towards action, because problems are never fully understood and are frequently changing. Often, the only way to learn is to try, and to allow teams to push ahead in the face of uncertainty, and to be brave enough to change if they are wrong;
• we need agreement on what to standardise and integrate such that the parts of the organisation experience standardisation and integration as empowering rather than constraining;
• we need the confidence to diverge from standards where this facilitates short term flow and we need the confidence to control divergence where this could hamper future flow;
• we need the honesty to adapt when we make mistakes, rather than retain attachment to our ideas.
• there are trade offs between optimising flow at a team level and at an organisational level. This requires judgement and appropriate alignment on vision and objectives.

Managing this flow is clearly about balancing multiple forces, which requires commercial insight, judgement and legitimacy. It is important to recognise that for almost all enterprises, maximising our ability to create client value rapidly is the goal, rather than minimising cost. If we succeed, simplicity and reduced costs will be a natural outcome.

Architecture is also fundamentally about communication. It is about actively listening to the concerns of clients and other stakeholders and about communicating the architectural decisions that are made at all levels of the organization towards achieving their goals.

The Good in architecture

Architecture is not merely about technical considerations. It is about a sense of purpose, a way of working that is rooted in vision, honesty and a bias to action. At its core it is a heartfelt commitment to helping others succeed in their endeavors.

The outcome is a platform

Any large complex business is inevitably evolving into a digital platform business, either as the provider of a platform (think Amazon), and/or as a participant on other platforms.

Understanding how we radically reconfigure our core capabilities into a digital business platform that is responsive to client needs is an engineering process (in the broadest sense). The configuration that an organisation will take is a strategic decision that is made at the top, but Architects have a fundamentally important role to play in outlining the art of the possible and in contributing a systematic engineering discipline to this process.

What kind of enterprise do we strive to be?

Freedom Charter

The 1950’s in South Africa were a dizzying time of cultural and political development. On the one hand, the crudest forms of the Apartheid system were being implemented. Every aspect of social, political and economic life, was codified based on a system of racial classification. Your race defined where you lived, the school you could attend, the job you could perform, the people you were permitted to marry, the services and entertainment that you had access to….

On the other hand, a new generation of leadership emerged in the anti-Apartheid movement that took a vigorous and activist approach to political opposition. This was the generation of Nelson Mandela, Anton Lembede, Oliver Tambo and Walter Sisulu. A generation of politicians, journalists, musicians, lawyers, academics and doctors emerged that shaped the country for the next 50 years, and ushered in democracy in 1994.

Amidst this maelstrom of activity, in 1955, various anti-Apartheid movements proposed to call a Congress of the People, that would define a charter for the country. A rallying call went out to all the people of the country:

“If you could make the laws, what would you do?” “How would you set about making South Africa a happy place for all the people who live in it?”

In his autobiography, Nelson Mandela describes what unfolded:

“The call caught the imagination of the people. Suggestions came in from sports and cultural clubs, church groups, ratepayers’ associations, women’s organizations, schools, trade union branches. They came on serviettes, on paper torn from exercise books, on scraps of foolscap, on the backs of our own leaflets. It was humbling to see how the suggestions of ordinary people were often far ahead of the leaders’. The most commonly cited demand was for one-man-one-vote. There was a recognition that the country belongs to all those who have made it their home.”

The submissions that were gathered by thousands of volunteers were formulated into the Freedom Charter that was adopted on 26 June 1955 at the Congress of the People, attended by 3000 people in Kliptown, near Johannesburg. The high aspirations that it enshrined inspired a generation and ultimately saw this high level of aspiration enshrined in the democratic constitution of South Africa (40 years later).

“The Freedom Charter captured the hopes and dreams of the people and acted as a blueprint for the liberation struggle and the future of the nation.”

The Freedom Charter sets out a blueprint that answers the question: “what kind of country would we like to be?”

This is an extreme example of a society in crisis that had the leadership to rally the majority of its members to formulate a blueprint that shaped its destiny.

The experience of the Freedom Charter teaches many useful insights that organisations can use.

• architecture is not an academic or ivory tower endeavour, it requires a lot of listening;
• it engages a wide spectrum of people in the organisation; people that understand client needs and their practical challenges and have a passion to redirect their efforts towards the common goal;
• it starts from a leadership vision / purpose, it isn’t just a free-for-all where any idea goes;
• it requires skill in coherently, synthesizing the core blueprint; not everyone’s idea can be incorporated;
• legitimacy is easier if there is a broad sense of having been heard;

What kind of enterprise do we strive to be?

Today, not every organisation has the same opportunity on the same scale as the extreme example of the Freedom Charter. But, at any scale, this can help us understand what architecture is about. It is an immensely exciting and inspiring endeavour. Fundamentally it contributes to the question: “what kind of organisation would we like to be?”

Most organisations have answers to this question in some form or another, but it is very rarely answered in a way that can translate directly into what Jeanne Ross, in “Enterprise Architecture as Strategy”, calls a “foundation for execution”.

Our overarching goal as architects therefore is this:

Based on the leadership vision for the organisation we strive to be, we define, as an ongoing activity, the configuration of the business platform, in a way that can be readily translated into a foundation for execution and an agile plan to develop it.

Too often enterprise architects are so caught up in concerns about technology or internal process that this important goal is missed. Starting with this goal can help to rapidly focus architecture efforts on the areas in which they will truly make an impact.

Architecture requires strong and visible leadership. It involves challenging the status quo and the assumptions that define it. It is exciting and vigorous; often dealing with pivotal, defining moments in an organisation’s life.

With the pace of change in the business, social and political world, almost all organisations need to grapple with these questions in some way. If they don’t, then they probably have little need for the type of architecture that we discuss. For a full exploration of some of the commitments that this entails, Ross in “Enterprise Architecture as Strategy” provides a systematic outline.

Bias to action

Alistair Cockburn describes architecture as:

Solving a problem that we do not fully understand, and that keeps changing underneath us

While working on our strategic intent and unpacking the implications of the “enterprise we strive to be”, we also need to let go of certainty, accept that we will need to adapt to changing circumstances and understand that this directs us towards a bias to action. The only way that we truly know something works is to try it.

Architecture as design

In recent years, architecture has been strongly influenced by design thinking and it is now impossible to do architecture without acknowledging this debt.

Design thinking

Design thinking is deeply rooted in understanding human problems.

We focus most of our effort on understanding the problem and don’t settle quickly for a superficial statement of the problem. We dig into the layers of assumptions and challenges that underpin the problem until we find the real insights into what is going on. We do this collaboratively and involve people from outside the organization.

We think creatively about the solution, free from attachment to what we have done before. If we struggle to find candidate solutions it often means we haven’t understood the problem.

We test solutions quickly with clients by building prototypes and experimenting.

We use data to measure what we do and to make decisions.

We focus on purpose and meaning rather than purely on financial value.

Architecture as design

There are probably people in your organization already working in this way and, like architecture, it is a practice that should be woven into the organistion at all levels.

There are obvious ways in which architecture can learn from and participate in design thinking, especially with regard to understanding the problems that we are solving for our clients, eliciting innovative solutions, and using the collaborative methods that make these efficient.

Structuring the architecture activities

Organisational structure

Should architecture functions be centralised or federated within the organisation? I believe that the answer is both: Architecture needs to be deeply embedded in each part of the organisation, where client value is created, because this draws in stakeholders and facilitates adoption. At the same time, architecture plays a pivotal role in connecting the organisation in ways that its structures often hamper. Questions around what to standardise and what to integrate cannot be answered in one part of the organisation alone.

The key role of the architecture team is to facilitate this process: to make sure that architecture is being done; that it is being woven into the organisational fabric; that it has impact where it is embedded and that it connects the organisation in ways that realise its strategy. It focuses efforts on the goals of the enterprise, while acknowledging that it must respond to creative tensions in the organisation. And it needs to be hands-on involved in doing the work, too, because we learn by doing.

As a general rule, the principe of subsidiarity should apply: decisions should be handled at the lowest accountable level to handle the matter. What this means in practice requires judgement, but it needs to start from a disposition of vision, trust, empowerment and mutual accountability

Architecture activities

Architecture activities happen at all levels of the organisation from the individual product team to the enterprise level.

We break the architecture activities down into three areas: creating architecture markers/artefacts, peer review and governance.

Below is a useful way of showing these two dimensions in a simple framework

While we need to make good decisions to retain the organisation’s ability to execute with confidence, we also need to recognise that there is no point at which every consideration is settled. Because we never have complete understanding of the problem and because the problem keeps changing, we need to be comfortable that the currently proposed architecture is our best view at a point in time, that it explicitly captures the inherent risks that come from uncertainty and that it evolves in a purposeful and inclusive way.

Architecture markers

At the heart of architecture practice is the day-to-day activity of producing architecture markers. A marker is anything from an idea that is discussed and internalised by a team, a whiteboard drawing or an iteratively designed blueprint. The important point is that this activity should be vigorous and continuous. Artefacts may be done by full-time architects, or they may be done by people in other roles who are actively part of articulating the architecture. Architectural markers should be captured as efficiently and visibly as possible (e.g. wikis).

A key goal is to enable the next effort

These activities can’t be effective unless there is a fundamental orientation towards client needs (rather than technology) and unless a representative range of stakeholders is engaged in the process.

Peer reviews

While the architecture markers are being produced, peer discussion naturally occurs, but there is space for some semi-formal milestones where the proposed solutions are presented and debated with a wider peer community.

Ultimately, the proposed architecture needs to be so well articulated that it can be told as a story, starting with the desired business outcome and working its way through a solution that is seen to be achievable and clearly meets our goals (templates often hamper rather than help this). Peer reviews are a good way to test that this has been achieved.

Architecture is story telling

Formal governance

The governance process should primarily be focused on making sure that executives are involved in the architecture process and are meaningfully accountable for the decisions.

Knowing that executive engagement is part of the governance process will also ensure that the views of stakeholders are well represented from the beginning.

In large organisations modern corporate governance requires showing evidence of effective architecture decision making, and this places specific requirements to demonstrate that the board of directors has appropriate oversight of the significant decisions that are made.

The essential elements here are to make sure that executives lead the architecture story and that key decisions are well communicated. This isn’t only achieved in formal meetings.

Putting them together

We have covered the continuum of architecture activities: from the vigorous construction of markers that is deeply rooted in a dynamic team environment (where we have access to a large capacity of smart people), through a healthy culture of peer review and finally cemented through organisational alignment, buy-in and commitment to a set of decisions that move the organisation towards its objectives.

An important role of an architecture practice is to coordinate all of this and build capability.

Managing the work

As much architectural effort as possible should be embedded in the teams that are accountable for the outcome. It should be managed on their backlog and the impact of doing the work should be well understood by the product / portfolio owner so that it is prioritized in a way that maximizes flow. We must avoid waste caused by big up front design that could rather be broken up so that it can be tested incrementally. On the other hand, we must avoid waste that comes from not having prepared for critical decisions and that leads to teams waiting for these decisions to be made or building solutions that are poorly considered and ultimately hamper the strategy.

It is the product / portfolio team’s responsibility to manage this delicate balance and to aim for just in time and excellent decisions. They should not see this accountability as belonging to a separate architecture team.

There will always be enterprise-wide decisions that cannot fit into existing structures. For these, there should be a specific architecture backlogs. These will be needed at different levels of the organisation.

Implicitly, this is a Kanban style process of focusing on the most valuable activities and measuring ourselves on value delivered. In the case of architecture questions though, this process is more extreme in its uncertainty and therefore even less suited to rigid planning than other aspects of product development. Equally, the cost of delay associated with making poor decisions (or not making decisions at all) is high. Managing these forces with insight and skill ultimately defines the best way to do architecture in a particular organisation.

Visibility of what goes on in these backlogs should be a healthy mixture of automation and face-to-face discussion.

Team skills

What kind of team do we need to facilitate architecture practice across the enterprise?

We need practitioners, who facilitate and participate in the architecture work, learn from each team and help each team to learn.

We need business specialists, who understand what we are trying to do as a business and how the business and industry function (and how they could function in a disrupted industry).

We need user experience designers, who understand how to frame the client’s problem and how to facilitate a collaborative process to truly understand these problems.

We need engineers, who understand how to build solutions and how to design technology platforms that support them. They need to understand emerging technologies and how they could be applied. They need to be able to test ideas with prototypes where necessary.

We need data / information architects that understand data modelling, data management and data engineering.

We need data analytic skills to bring a culture of measurement to the decisions we make.

We need drivers, who make sure that work gets done, that it is done collaboratively and that we balance the bias to thinking that exits in many of the other roles.

We need communicators, who collect and share information and make sure that it is evangelized across the organisation.

Good architects should be able to do most of these roles. In practice we recognize that a core architecture team is likely to be small, diverse and multi-skilled, and closely working with a much broader virtual team across the organisation in solving specific problems, and connected outside the organisation to bring insights and skills that we don’t have.

Architecture process

This is a typical process that I have found to be effective in complex organizations:

• The primary stakeholder requests guidance on a specific problem and articulates an objective.
• A small team of people is assembled who best understand the business and technology aspects of the problem.
• The team identifies all stakeholders to the problem and interviews each to understand their roles, their challenges and their objectives. They dig deeply into the challenges.
• This information is collated as a high level view of the current functions (the business operating model), an aggregated list of challenges and an aggregated list of objectives. These are tested regularly with stakeholders.
• A list of possible actions that can address the problem and challenges is created. This covers a range from short term tactical improvements to strategic changes.
• The strategic business and technology platform is articulated at a relatively high level.
• Through the process, the outcomes are played back to stakeholders, knowing that all objectives of all stakeholders cannot be met, while still getting buy-in from all stakeholders. Actions are prioritized.
• The outcomes are agreed with the primary stakeholder, who should have received regular updates and given regular feedback through the process.

This process needs to be adapted for each business challenge being explored. There are no strict templates.

Culture and change management

Getting an organisation to respond to the efforts that have been described is actually a Herculean feat. These efforts can’t succeed without executive support. There is very little specific advice that can help with securing this, because all organisations are different. Some general ideas for getting this support are:

1. Be deeply rooted in achieving the business outcomes, whether this is on a small or a large scale. Show a profound understanding of these business outcomes. This requires strong business, financial and human acumen.
2. Achieve legitimacy by making an impact, and use this legitimacy to build support for other architecture efforts. This always requires having a good team of people around you — both your core architecture team and a wider network in the organisation that guides and supports your endeavours.

Depending on the team and the state of the organisation, it may be possible to start at a grand enterprise scale, or it may be better to break the problem down and show high impact results in a particular context and then grow from there.

A systematic way to think about how to achieve the level of co-operation that you need is provided by Clayton Christensen’s “Tools of cooperation and change”

He outlines two areas of agreement that one needs to assess:

1. The level of agreement on what we want to achieve, and the implicit values, priorities and trade offs involved in this
2. The level of agreement on how we will achieve it, that outlines the journey and the means we will use

Both of these are central to the architecture process, and it is especially important to be always conscious of where we stand on these agreements and how this influences the activities that we perform.

Christensen identifies four tools that we have to change the organisation (Power, Management, Leadership and Culture). It is worth reading his paper for practical ideas on how to manage change.

Architecture governance through IT automation

So far we have focused on how the architecture process works to solve specific problems, but we also need views of the current architecture and how it is evolving. This is often a painful and bureaucratic process and is very rarely kept up to date.

With modern engineering practices, it is becoming easier for teams to take accountability for this and to automatically capture information on how technology components realize business capabilities, how components depend on each other, how applications are deployed and the changes that have been made or are planned to be made to applications.

There is some co-ordination needed to make this work, but it is very feasible to have teams do it as part of their engineering practices, integrated into their backlogs, issue tracking systems, deployment automation, monitoring and testing.

Focus on the near and far, not the middle

When striving towards our vision and our strategic objectives (the far), we are actively engaged in doing work to realise it (the near), while having to plan a route to connect the two (the middle).

In my experience, we spend (and waste) too much time on the middle. Ironically, this is the area of least certainty, because the far is well understood at a high level, relatively slow changing and far enough away that changes can be purposefully managed. The near is also well understood, because we are doing real work and learning.

Most of the waste is in the middle and there are two good examples of this waste. The first is detailed multi-year plans and programmes, which should be replaced by high level stories of the steps we currently believe we will take. These high level stories should be updated in a purposeful way as we learn and as client demands change.

The second is building standard foundational capabilities (for example large scale infrastructure, data or channel capabilities). While it is critical to have a vision and blueprint for these foundational capabilities, no-one in the organisation should be waiting for them to be built, we need to allow them to be built out in prioritised pieces as we build business capabilities. This allows us to learn quickly and to evolve our vision for these foundational capabilities in the context of real business problems.

One way to express this is:

We have a vision and blueprint for horizontal foundational capabilities, but we always execute against vertical client capabilities

The challenge ahead

While some of the ideas presented are aspirational and potentially difficult to implement in large organizations, I believe that they should not be seen as a burden that will drain scarce resources. If anything, they should lighten the load on the organisation in the short term, creating alignment, efficiency and flow.

What they do demand, though, is resilience, collaboration, and a team with a strong desire to support stakeholders in their struggle to progress.

In a future article, I will explore some practical ideas that large enterprises can reconfigure themselves into digital platform businesses.