Whilst replying to a comment on my post on Comparing Roles I made reference to a concept of relating software engineering to the design and build of lighthouses.

Upon this point I wish to elaborate further as this is one of many blog posts that I have been meaning to write for a while.

The concept, which I call Lighthouse design, is something I first considered about two years ago. It’s not a new concept, a lot of engineers talk of similar principles, but I’ve never seen anything concrete written down which explains it in quite the same way.

Eddystone Lighthouse

For very good reasons I am going to use what is probably the most famous lighthouse in the world as the basis for this post. Firstly, this is a lighthouse that has been rebuilt at least four times by different engineers since it’s first incarnation in 1696. Secondly, whilst the design has changed on each occasion, the basic principles of the construction have remained the same. As is the case with all lighthouses, it serves a single purpose, to warn shipping of the hidden dangers brought about by the high tides. To put this into business terms would be to say their service is a warning and their products are light and sound.

For those of you who haven’t read my previous article and the related comments, you may be wondering what lighthouses have to do with software engineering or why, out of all possible architectural constructions, it is this which I personally use for analogies of system architecture.

Choose your platform wisely

When we consider the basic infrastructure of a system there are primarily two platforms upon which we can build. There are *nix like system such as Unix, BSD and Linux; and then there is Windows.

For the purposes of this analogy, *nix type systems should be considered to be bedrock as these platforms are the most stable operating systems in the world.

The bedrock of any lighthouse is also the place where it is the most effective. You don’t build a lighthouse where the seas are clear, this is a waste of resource. Instead you place them where the danger lies. In order to do this, you need a very strong foundation.

When designing any large scale application, the very first thing that must be considered is the environment. Where is this application going to run? How strong is it? What can you feasibly throw at it before it starts to become unstable?

Standing nine statute miles off Rame Head in Cornwall, the Eddystone lighthouse protects shipping from the Eddystone rocks to the SSW of Plymouth Sound. Whilst the first two iterations of the lighthouse are noteworthy in their own right, the particular version of this lighthouse I wish to start this analogy with is the third construction which is also known as Smeaton’s Tower.

Actually if we take a step back for a moment, we could consider the first two iterations of the tower to be early versions of software. Nobody get’s it right first time and only by looking at the designs which have failed can we hope to build something which will succeed.

The development of any architecture begins with the selection of the environment in which it is due to operate. As already described, for lighthouses this is the most prominent place where it may serve as a warning without becoming unaccessible; whilst for software, this is the most reliable platform suitable for the purposes of the application.

I chose Smeaton’s Tower as the example for this article because this is a great example of a construction which literally outlasted its’ foundations. Even today the base of the tower survives out at sea, just beyond the current lighthouse, where it had proved to be too strong for the Victorians to dismantle.

Within our analogy, we have now recognised that the foundation of any software application is the operating system. For an application, we don’t really care about the hardware underneath but the operating system is critical to the success of our system.

Once we have chosen our foundation, the next stage is to design an application which works with the platform to achieve the desired results. Depending on the business model, this result will vary but the design of the application must be achieved in such a way that it works with the landscape and not against it.

Design for the environment

Taking the construction of Smeaton’s tower as the example, once the foundation had been chosen, the next phase would be to choose a design for the structure which would stand the test of time. For this, Smeaton chose an Oak tree and designed it in such a way that it worked with the rock upon which it was being built.

Smeaton's_Lighthouse00

Of course within software, it is impossible to build an oak tree as a service but we can still follow the concept.

The first thing that will be put down will be the roots. For software this should be any hooks into the operating system, for example file commands, system calls and memory access. These absolutely must be tested first. Remember, just because a foundation looks stable, it doesn’t mean that it is. Even the most tried and tested platform becomes unstable over time. This is erosion and it’s a natural part of the engineering cycle. Accept it, work with it and underpin where necessary but also recognise that one day, you’re going to have to move.

Understand the strengths, and the weaknesses.

As engineers we hear the term “DevOps Engineer” bandied around occasionally but every engineer should be a “DevOps” engineer. Every engineer should not only know the applications they are developing inside out, but also the platforms they are developing for.

Only by understanding the nuances of the environment can we ever hope to develop applications which stand the test of time.

Recently I posted about the glibc ghost vulnerability which affected Linux platforms. This serves as a fantastic example of a flaw in the bedrock which went undetected for a number of years because nobody thought to test it.

When you’re testing a foundation, it’s a destructive test. There is every possibility that once you step onto that particular ledge it will collapse under your weight. If it doesn’t, great but that’s only a small step in the journey.

When an architect examines the landscape in which a building might be constructed he takes a team with him including a geologist and an environmentalist. The geologist is looking for any flaws in the ground which may topple the building they wish to construct and the environmentalist is looking for impacts your structure may have on the existing environment.

Within this scenario, the geologists of software architecture are infrastructure engineering and the security team. These are the people responsible for understanding what the foundation can support and this is where the critical flaws are likely to be found which could jeopardise the stability of your application. These are most likely to be security vulnerabilities within the operating system itself and it is these which will have the worst effects on the business application of your software, and as we’ve seen countless times before, these can be catastrophic.

The environmentalist of any engineering project is the business analyst. They tell us the story of how we need to build the application to have maximum effect with minimum impact.

Choose your materials wisely.

When an architect is designing a building he looks at what materials are available to him locally which suit his particular design whilst offering strength and resilience to the forces he wishes to place upon it. If he can’t find the right material locally which can take the stresses and strains of a full blown ocean storm, then he will look further afield.

For Smeaton the choice was relatively easy without having to import far. Granite is common to the Devonshire and Dorset coast, and is an incredibly hard rock which can take the strain of the ocean with relative ease.

For software engineering, the most obvious comparison for materials would be the programming languages and each language is suitable for a different purpose. For example just as you wouldn’t build a lighthouse out of sandstone, likewise you won’t build a large scale infrastructure out of Javascript. This isn’t to say that you can’t do this but if you do, don’t complain when it all falls down.

The choice of material is very much subjective to the environment upon which you wish to run the application you’re designing and the amount of stress you are likely to place upon it. For example, the design of a large infrastructure is better served using a flexible byte-code compiled language such as Java or C-sharp whilst middle-ware web applications may be better served in PHP, Ruby or Python.

The mapping of languages to their respective materials is highly subjective and everyone will have a different opinion on this but for the purposes of this analogy, this is how I personally would map some of the common languages to what I see as their respective construction materials.

Language Material Description
C Granite C is probably the strongest language for application and server architecture. Tough and resilient it will, if cut properly stand up to anything you throw at it. It’s drawback is that this toughness makes it a remarkably difficult language to work with.
C++ Limestone A useful language for building applications but not really strong enough for large-scale infrastructures.
JAVA Steel Fantastic for applications and backend architecture alike. Java is a very versatile language but is prone to rust if left unattended.
C# Marble Re-calcified limestone. Has a beautiful grain to it but it’s brittle if hit from the side. Works best when used as decoration. Marble is malleable and relatively strong. This malleability makes it a very versatile language.
SQL Concrete / Cement No application can exist without a data-source and the most common datasource comes from databases. It’s the data which binds everything together.
PHP Brick The workhorse of the internet. You can build very elegant structures with it but when exposed to the elements will rot if not well maintained.
Python Gritstone Python is a very gritty language which is easy to work with. You can mould it very easily to suit a specific requirement. When polished it has a wonderful grain to it and even if left un-hewn it has a rugged beauty which blends into any environment.
HTML Timber Although not a real language, HTML is essential for constructing facades to web based applications.
Javascript Wood / Glass Can be used at server-side as an infrastructure supplement but works best when used to illuminate things in the browser.
CSS Paint / Decoration Not a real language but really useful for putting a pretty face on a design.

Once we have chosen our location and materials, the next phase of the architecture is to draw out an artistic representation of what the finished building will look like. This artistic representation gives the client an understanding of what they are likely to expect as well as being an early indication of whether the finished construction will blend in nicely with the environment.

Within any web-based service industry, this initial design is carried out by the web-designer who will produce wireframes of the architecture, showing how each component fits together. This is a representation of the service and products on offer. Once the wireframe, which will probably be black and white, has been agreed, the designer will trot off back to his colouring board and colour it in to meet the requirements of the client.

Design for strength.

When Smeaton built his lighthouse he pioneered a completely new technique for bringing strength into his building. The Eddystone rocks are notoriously dangerous and subject to incredibly powerful waves coming in from the Atlantic ocean. Smeaton would have known that normal bonding techniques used for stone would not be sufficient to stand up to the stresses and strains brought about by an ocean in full storm.

Instead, he designed a new interface which would help spread the force of nature around the whole building by interlocking the granite blocks using a dovetailing technique to interlock the stonework and secured them with marble doweling.

Smeaton Dovetailing

 

Before designing any application, think very carefully about what the object is, what it’s properties are, what it accepts and what it provides.

If we were to design a class to represent a single granite block from the above diagram, we first have to understand what a granite block actually is and how it fits into Smeaton’s design

granite-block

Now, ask yourself this. Do I really need to know the composition of a granite block?

I’m guessing at this point you’re all saying “No”, and you’d all be absolutely right. There is no need for an engineer to know the intricate details of every component found within a given architecture. Within this scenario, this is the job of the stone-mason. The job of the engineer is to take a finished component (the block) and interact with it.

Within software engineering, we call this “black boxing”. We take a third party component and use it within our systems without caring about how it achieves its’ goal beyond a visual inspection (aka, does it do what it says on the tin?).

Within our lighthouse there are a number of components whose composition we don’t actually care about, the stonework, the glass, the timber, metal, etc. As engineers all we’re looking to do is assemble them in a recognisable format.

Within UML, if we looked purely at the outer shell of the lighthouse, this might be modelled as such:

lighthouse-structure

Build for stability.

It seems fairly apparent from the design of the lighthouse that Smeaton understood the properties of Granite, the nature of the forces which the ocean capable of raising and the type of structure which could withstand these forces. This kind of foresight is something I believe we lose when faced with the demands of generating software in a high pressure environment.

When Smeaton built his tower, there was no way to predict how quickly erosion would have an effect on the foundations of his lighthouse. Using the most pioneering techniques at the time, he designed a tower which was strong, selected a foundation presumed stable, and placed it in the most prominent position available.

Smeaton’s tower has outlasted it’s foundations by 138 years. Despite being dismantled and moved to a more secure location, the tower still stands whilst the original foundation continues to erode away.

Lay your delivery pipeline.

The stone for the lighthouse was to be imported from Portland in Dorset which is about 130 miles up the coast from Rame head. In order to get the stone to the lighthouse it was to be brought by ship.

During the time the lighthouse was being constructed, France was at war with England, the sea’s around the south coast were awash with French privateers and at least one delivery was delayed because the Royal Navy press ganged sailors into service.

Like any software delivery pipeline, there is always risk involved in getting artefacts from one location to another. Ok, I don’t expect you to have problems with French privateers interrupting the transition and testing of your software but if you design the pipeline badly you’re going to have other problems to contend with; be it deployments, integration or execution.

The delivery pipeline for code takes you from commit into version control, through CI and all the way out to live. I will cover this off in more detail in a later post as this is a topic unto itself.

Put down your roots.

When designing any system it is worthwhile paying attention to the architecture of buildings. Clients always want to see progress at the earliest opportunity. They don’t understand the backend of the system or appreciate the amount of effort which is required in communicating between the different layers and services required to build up a complex architecture. Because clients always want to see tangible results at each review, it becomes very easy to fall into the trap of building from the top down.

Whilst in some instances this may be a realistic approach, in practise it is never a good model to follow and will inevitably lead to an unstable application.

As I stated earlier in this article, your foundations should work with the landscape instead of hacking at the platform to achieve the goal of your application.

The rock upon which Eddystone lighthouse was initially built is regularly under water. This meant it would be impossible to drill down into the rock or cut it away to present a flat and level surface upon which the lighthouse could be built without unnecessarily risking the lives of builders out at sea.

To overcome this, Smeaton developed a new form of cement known as Hydraulic Lime which could set under water. This cement was then layered on top of the rock to build up a platform for the lighthouse to stand upon.

Lighthouse Foundation

When building upon your foundation, there are two lessons here.

The first lesson is to not be afraid to try something new. Just because something has been generally accepted as being the right or correct way to do it, doesn’t mean that it is. Innovation comes from breaking with tradition and in doing so, you may find the new norm.

The second lesson to be learned is that whilst it can be an idea to trim unnecessary services from your system (this is simply good practice, the equivalent to sweeping loose rock from the base), you should never alter the underlying structure in any way. Altering the structure can introduce weaknesses which speed up erosion or create flaws which can be exploited and lead to the destruction of your application, or the loss of customer data or worst.

Construct your product.

Once the foundation has been prepared and levelled off to a suitable condition, we can now start building the application.

Software and lighthouses alike are designed and built in layers. At the top is the presentation layer. For lighthouses, this is the light room, for your application this is the graphical interface or display mechanism.

Below this is the business logic layer, this ties into the living quarters for the lighthouse. Below this, the data access layer, the store rooms of which there were two in Smeaton’s tower. Finally there is the data source. This is the entrance through which supplies can be brought in.

software_layers

Hand over.

Once the construction has been completed, the lighthouse or application can be handed over to the clients and put into service. This is the equivalent of a lighthouse keeper and within Software engineering, this is the operations team. It is their responsibility to ensure the smooth running of the application and to call engineers out for routine maintenance.

Just as a lighthouse keeper will, as the building beds in, come to know the lighthouse as well as, of not better than the architect, over time, the operations team should come to know your application better than the team which built it.

They will be dealing with it on a daily basis, they will be able to identify where the problem areas actually are and schedule maintenance around it. Their understanding of the live platforms and services should be used to help drive the next generation of application to be designed and built and their expertise in the operation of platforms and applications alike are key to the smooth operation of the business as a whole.

Routinely maintain and upgrade.

Over time, components wear down. Erosion from the weather and ocean attack the walls of the lighthouse, glass breaks and mirrors tarnish. These parts are expected to be replaced on occasion and are fitted in a way that they can easily be accessed and new components put in in their place.

Likewise, as technology moves on, new mechanisms for lighting are developed or there may be a desire to install new features (fog horns for example). Even lighthouses have to be extensible.

Smeaton’s lighthouse would have been considered to be too small to install the latest technology as new machinery became available but had it remained in place, the advent of electricity would have resolved these problems.

Upgrades to software are often inevitable. It is highly unlikely that a perfect software product, which requires no upgrades will ever be designed but the basic overall architecture should remain the same and never need to be completely overhauled.

Conclusion

There is one final lesson which can be learned from this iteration of the lighthouse.

When Smeaton was building the lighthouse he noted that the rock upon which it was being constructed was eroding and likely to one day endanger the entire structure. Once this had been understood, he made an offer to underpin the rock at a cost of £250. This offer was rejected.

121 years later, erosion to the platform underneath meant the tower had to be moved and a new one constructed. The cost of moving a lighthouse and constructing a new one would have run into the tens of thousands. To put this into perspective, the original cost of Smeaton’s tower when it was built in 1759 was £40,000. This equates to £5,346,237 with inflation. Whilst the cost of moving the lighthouse would have only been a fraction of that amount, this is money which could have been saved if the business owner had simply listened to the Engineers in the first place.

Of course they never do.

4 thoughts on “Lighthouse design

  1. Jeff

    A great analogy, and interesting to learn so much about building lighthouses! I often find that building an application top-down can work well. It helps expose the necessary calls into the mid-tier. They can be mocked at an early stage, and then used to guide design comms between the layers. Sometimes, building from the foundation up doesn’t get you where you want to be, and in this regard, software engineering is different from building. Trying to build a lighthouse from the lightroom downwards would probably have been more tricky.

    1. Whilst I get what you’re saying on this, I have to wonder how much of this ‘top-down’ mentality really requires a mindset shift.

      Coming from an environment in which product owners want to see tangible results at every opportunity it is very easy to fall into the trap of building the roof first, but if you don’t know what the walls or foundations are going to be composed of, or even what shape the walls will take then there is every probability that the roof you have just built won’t even fit properly; or, if by some chance it does, the underlying structure will buckle and break with time.

      If we are finding ourselves in the position where we have to build from the top down “because it aids in the guidance of comms between layers”, then the question comes back to the quality of the original design. The top down approach is equatable to starting out with a concrete block and saying “Actually that block is now granite”. The composition changes and shifts continually because the next person to come along might decide that the concrete granite block is now actually marble. Now it’s a concrete granite marble block. Well which one is it?

      Within architecture, whether it is building architecture or software architecture, design comes before build. In building architecture, every nuance of the composite of the building is detailed. You know what to build it out of, what stone, what type of timber, even the colour of the glass and fixtures long before the foundation is dug. You know what the building will look like from architectural drawings and artistic representations. At that point the building really becomes an assembly exercise, taking components off the shelf and trimming them into place.

      The top down approach to building applications should really be avoided unless you don’t care too much about the overall stability. Top down is too ad-hoc to last and becomes a maintenance nightmare over time. Why? Because there is a tendency to build first and draw the design later, which inevitably leads to the design never being completed.

      Top down should only be used for proof of concepts. “I know I need to build something but I’ve no idea what it might look like”. Think of this like Smeaton designing the hydraulic lime cement. He had no idea what he was going to get, he just knew the current technologies were not suitable for his requirements. In order to develop the HLC he would have likely mixed components together until he found what he was looking for and then incorporated it into his design.

      Top down isn’t a wrong approach as a concept but it’s certainly the wrong approach for stability within an application if used to build the application. And if you do find yourself building top-down, ask yourself why because it very likely means your design is incomplete.

  2. Phil Marsey

    Came across this and thought this would be interesting food for thought.  🙂

    http://www.developerdotstar.com/mag/articles/reeves_design_main.html

    Hope you enjoying you new role.

    Phil

    1. Hi Phil

      Firstly, sorry for the delayed response to your comment – the last few weeks have been phenomenally busy with one thing and a renovation. To answer your question first, the new role is a roller-coaster. They don’t really know what to do with me so I get a pretty good run at R&D showing them what they didn’t know they needed 🙂 Definitely positive but no Plusnet.

      Definitely interesting articles. I did come across them many years ago when I first started development but back then I didn’t really understand the principles and promptly forgot they existed. Nice to be pointed in their direction again.

      I seem to recall that I still owe this blog a post (From Well to Refinery) – will have to get this one written up soon and based on recent experiences, I’m pretty sure there are a lot of posts to follow.

      Hope things are well with you. I hear you and Mr Lloyd are starting good things.

Leave a Comment