Vestigial Technology

Semprini
Dec. 20, 2018
- 1. IT Architecture
- data
0

This blog explores the rationale which has led me to create the data strategy Data Autonomy. As a consultant to corporate IT for over 2 decades I see the same patterns and challenges again and again. The common thread is how IT becomes slower at responding to the business due to how data is handled.

I'd like to start with a quote from a renowned scientist - Dr. Bunsen Honeydew - "Here at Muppet Labs, where the future is being made today."

In regard to corporate IT I would amend this to "where future legacy is being made today." I.e. All new business systems are tomorrows legacy systems due to the constant increase in business change.

Architecture at all levels is changing. The two biggest drivers to the way we architect enterprises and solutions are the proliferation of targeted cloud based services and overcoming the drag of legacy systems.

"...the real force behind the success of rapidly growing firms is rooted in their ability to adapt." - Jim Hatch, Jeffrey Zweig.

Key to the ability to change in a distributed landscape is effective management of significant business data.

Legacy Transformation

Corporate IT has evolved much like ourselves - as Darwin stated 'we bear the indelible stamp of our lowly origin' and IT has it's fair share of appendixes and male nipples.

The reason we still have vestigial body parts is dependencies. The evolutionary process takes aeons to sideline redundant functionality because all our systems are interdependent. This is very similar to how IT has become a lumbering mass of technology.

If dependency is the enemy of agility then the architecture focus needs to concentrate on limiting dependency.

The time between identifying a business need and delivering the required IT solution needs to become hours and days rather than months and years. Delivering this cadence must not introduce dependencies so we can continue to deliver again and again.

The most important responsibility of IT becomes enabling accuracy and access to significant businesses data.

Dangers of Shadow IT

"Most organizations grossly underestimate the number of shadow IT applications already in use,"

- Brian Lowans, Gartner.

Shadow IT can expose an organization to a host of data privacy and security-related compliance risks. "A data breach resulting from any individual BUIT purchase will result in financial liabilities affecting the organization’s bottom line. Liabilities can be very large due to a mix of costs that include notification penalties, auditing processes, loss of customer revenue, brand damage, security remediation and investment, and cyber-insurance." - Brian Lowans, Gartner

Bypassing IT to procure cloud services can also leave an organization in violation of regulatory compliance requirements such as the Payment Card Industry Data Security Standard, the Control Objectives for Information and Related Technology and the Basel II international standards for banking.

At some point in a firm's growth, the lack of practical policies toward shadow IT and adoption of unrestrained cloud services will fragment significant business data and become a significant drag on operational change.

Organizations which do not adapt to this distributed IT landscape and embrace some form of shadow IT will be less flexible and less able to compete. The cadence of change in an organization is key to it's growth.

Data

Applications are built to perform business functions and, to do their job efficiently, they rightly structure and store their data for that purpose. Applications and application vendors do not have a holistic view of how your business operates as they service multiple businesses or industries. Dependencies on the data sourced from applications which comes with a host of application idiosyncrasies is why deprecation of legacy systems is notoriously difficult. Issues managing dependencies are also manifested in integration and can lead to a proliferation of middleware layers and versioning as soon as regression testing becomes too complex.

The standard way of approaching data access is to expose and integrate the data held in various applications through middleware layers/ESB. Thus, in theory, de-coupling the back-end from the consumer. As business progresses and the data layer loses that new ESB smell, it becomes more and more clear that there are other forms of coupling which are slowing down the cadence of change over time.

Versioning seems to be the default position for handling change. This is a false economy. Under the pressure to keep time frames and budgets, the first casualty is upgrading old interfaces. I am yet to see a business case where upgrading old interfaces to keep an n-1 versioning strategy stacks up. Jamie Zawinski put it succinctly "Some people, when confronted with a problem, think "I know, I'll use versioning." Now they have 2.1.0 problems. ". Every version is another set of dependencies which increase the cost of changing, upgrading or replacing an application.

The more dependent we are on an applications data, the more inflexible we are in changing the application.

My solution to enable large enterprises to act as a startup is discussed here Data Autonomy Overview