Devops and The Principle Of Flow

lean-software-development-1-728

In the technology value stream, work typically flows from Development to Operations, steps consisting of functional areas between our business and our customers.

As stated in the lean principles developed by Toyota, we should optimize to get a single-piece fast and smooth flow for our releases.

We increase flow by:

  1. Making work visible,
  2. Reducing batch sizes and intervals of work
  3. Building in the quality, preventing defects from being passed to downstream work centers.

Why a fast flow is needed?

By speeding up the flow through the technology value stream, we reduce the lead time required to fulfill internal and external customer requests, further increasing the quality of the work while making us more agile.

Our goal is to decrease the amount of time required to deploy the changes into production and increase the reliability of those services.

Make our work visible

agile-pm-kanban-board

A significant difference between manufacturing and technology value streams is that our work is invisible.

It’s so easy for work to keep bouncing off between teams and yet have no visual control over it.

To prevent this and make out work more visible, we can use something like a Kanban board. (I prefer Trello for this).

Ideally, our Kanban board will span the entire value stream, defining work as completed only when it reaches the right side of the board.

Work is not done when development completes, but only when our application is running successfully in production.

Limit Work In Progress (WIP)

In technology, our work is far more dynamic than manufacturing. Teams have to satisfy demands of multiple stakeholders. As a result daily work gets dominated by urgent requests for work coming through every communication channel possible.

We can limit multi-tasking by using Kanban board, such as by codifying and enforcing WIP limits for each column on the board.

For example, we may set a WIP limit of three cards of testing. When there are already three cards in the testing column, no new cards can be added.

Using Kanban ensures that work is visible and WIP doesn’t get piled up.

Reduce Batch Sizes

one-piece-flow

Another key component to creating smooth and fast-flow is performing work in small batch sizes. Prior to the lean manufacturing revolution, it was common practice to manufacture work in large batches.

However, large batch sizes result in skyrocketing levels of WIP. According to lean principles, the ideal is a single piece flow, where each batch size is of just one.

Let’s take an example:

Suppose we have ten brochures to mail and mailing each one of them requires 4 steps:

  1. fold the paper
  2. insert the paper into the envelope
  3. seal the envelope
  4. stamp the envelope

Now in the traditional batch processing flow, we will perform each step sequentially for all ten envelopes.

In the lean one-piece flow, only one envelope can be at any given step. In other words, we fold the paper, insert it into the envelope, seal the envelope and stamp it before starting the next one.

How is one-piece flow dramatically better?

In the above example, suppose each step takes 10 seconds. In batch processing, we get our first complete envelope after 310 seconds, but with the one-piece flow we get it just after 40 seconds.

Worst, what if we find that the way we have folded the paper, doesn’t allow the envelope to be sealed. In which case we’ll be in a bigger trouble?

Eliminating hardships and wastes in the value stream

According to the Toyota Production System pioneer Shiego Shingo, a waste is:

The use of any material or resource beyond what the customer required or is willing to pay for

In software development value stream, a waste is anything that causes a delay for the customer, such as activities that can be bypassed without affecting the result.

The following are some common categories of waste that we encounter when implementing lean in software value stream.

  1. Partially done work
  2. Extra processes
  3. Extra features
  4. Task switching
  5. Waiting on QA or testing or acceptance testing
  6. Defects and bugs
  7. Non-standard or manual work

Explaining each of the above point deserves a post of its own. Will do that soon.

Conclusion

Improving flow through the technology value stream is essential to achieving DevOps outcomes. We do this by making work visible, limiting WIP, reducing batch sizes and eliminating wastes from our processes.

All of this will allow us to become more agile and will help in reducing lead times dramatically, and at the same time increasing the quality of releases.

That’s all, folks!

 

Advertisements

Practical Problem Solving Framework: Inspired By The Toyota Way

toyota-7-step-pratical-problem-solving-process

We all will agree to a certain point that having a system/process for anything reduces chances of errors.

As an engineer or someone people look forward to propose solutions to problems it’s beneficial to have a framework in place to solve problems effectively.

Recently I was reading The Toyota Way, and it suggested a framework to Practical Problem Solving. It almost felt trivial that this sort of framework would be invaluable to software engineers too (in fact for everyone).

When confronted with a problem, first we want to make it crystal clear and get a grasp of the real point of cause. That’s followed by a series of 5 WHYs? to investigate the root cause. And finally countermeasures, evaluations, and countermeasures.

1. Initial Problem Perception

Large, vague and complicated problem is presented. The first step is to perceive all the information available at this point information of time.

Ex. “Hey! Metric X is showing incorrect value”

This doesn’t show the actual problem, but just a perception of how some internal user saw it.

2. Clarify The Problem

Next step is to clarify the problem to scope it down. Go and see the problem yourself. Analyse it and get a clear understanding.

As you are seeing the problem first hand, we want to gather as much information as possible.

Ex. So the entire analytics data was actually not consistent.

3. Locate Point Of Cause

Next step is to dig a little deeper and try finding the point fo cause.

Where is the problem observed? Where is the likely cause? This will lead us to the vicinity of the root cause, which we find in step 4.

Ex. Analytics system is working correctly, just that it sometimes doesn’t get updated every 5 minutes like it’s supposed to.

Here we rule out other possible causes, like a bug in the code or wrong data was tracked in the first place.

4. Ask, 5 WHYs? Investigation of  the root cause

Here from the direct cause, we expose and go deep to the root cause of the problem by asking WHY five times.

Ex.

1. 1st why – Why was data inconsistent: Because analytics didn’t get updated on time.

2. 2nd Why – Why analytics were not updated on time – Because scheduled ETL jobs didn’t run on time.

3. 3rd Why – Why the schedule jobs didn’t run on time – Because CPU usage was 100%

4. 4th Why – Why CPU reached 100% – Because server instance size was not enough to handle increased number jobs.

5. 5th Why – Why server size was not enough to handle the spike in usage –  Because our auto-scaling is slow.

By asking a series of 5 whys, we can generally get to the root cause of the problem and fix it there instead of just duct-taping it and be waiting for it to rise again.

5. Countermeasures

This step is fixing the root cause of the problem so that this doesn’t come up again.

Ex. Moved to a more sophisticated auto-scaler to manage spikes in usages and setting up alerts to monitor the performance.

6. Evaluate

After the countermeasure have been executed, it’s important to evaluate the effect post that. Was the problem solved?

Ex. “Now analytics are always in sync and even if they miss getting updated, we get an alert to know it beforehand and take action.”

7. Standardize

This resonates with another Toyota principle of jidokameaning building in the quality.

How can we standardize the countermeasures such that similar problems are not faced again? How can we propagate our learnings across the organization?

Ex. “Document and standardize the process that for all our instances and jobs proper alerts must be in place so that we know when they are malfunctioning”

Conclusion

This was my take on how we can learn from a cross-discipline organization like Toyota on how to have a process and framework in place to solve problems effectively.

Afterall, problem-solving is supposed to be fun and having a proper framework in place, helps us keep it that way!

That’s all, folks!

 

8 System Design Principles I learned After Doing It Wrong More than 50 Times!

laptop-3190194_960_720

 

At Squad, we strive to build awesome products to solve customer(internal and external) needs. As a product engineer, paramount part of your job is to design and build products. Dig deep into the root cause of the problems, design solutions and implement them as the end product.

Over the course of my journey so far, here are the 8 system and product design principles that I’ve learned from other awesome people at Squad, from feedback and simply doing it not right enough multiple times.

1. What is the underlying problem that led to the feature request?

At Squad, you don’t just code the requirements into the software. As a product engineer, it’s your responsibility to remove the layers and expose the root problem that led to the feature requirement.

Get to know the root cause of the problem that you are trying to solve. Or even better, as the lean principles say “genchi genbutsu” i.e go and see it yourself.

2. How can you make the feature more robust, reliable and usable?

Once the essential feature requirements are finalized, we must press on how can we make the feature more robust, reliable and usable?

Things to ponder upon and take into consideration can be :

  1. The persona of the users that’s going to use that.
  2.  Scenarios in which that feature would be used. Ex, if in the case of fires, than show more data than needed for faster resolutions.
  3. Building in the quality in the product itself or “Jidoka” as said in lean.

3. What is the first iteration going to be?

Given the time and resources you have., what is the best possible first iteration of the product going to be? If it’s a large system or something you are building from scratch, there are always going to be iterations.

The main idea here should be to move fast and get things shipped. Good enough and shipped on time is always better than perfect and in-development forever.

4. How easy will it be to make iterations on the current feature?

The design should incorporate all the non-functional requirements to make future iterations easy.

Scale the feature? Change a component? Use a different 3rd party service? Your implementation should be flexible enough to incorporate and encourage these enhancements.

Design patterns are your best friend here.

5. What are the potential bottlenecks with scale?

Scale-land is where everyone wants to be, but it is scary. It breaks what was not supposed to break and has witnessed more horror stories than a haunted castle.

What are the potential bottlenecks that are not a problem now, but will break at 5X, 10X or 100X scale?

List them down on the feature ticket, or better document it in the code itself.

6. What’s the data that has to be captured and how will it be consumed?

Every feature in the product will need some data that needs to be captured to track it. It can be but not limited to:

  1. Action logs.
  2. Event logs.
  3. Metrics
  4. Failures.
  5. Anamolies.

What affects this majorly is how that data will be consumed? Store it in a structure that will make the consumption of data easy and efficient. Afterall, the only motive to store data is to use it.

7. How good the developer experience will be when interacting with the code base of that feature?

There can be many developers who’ll use or modify the code that you are going to write.

How will be their experience when doing that? Ex. Will the test cases you wrote, make them feel confident enough to make changes fast?

Few points to consider:

  1. Is the code well documented?
  2. Are test cases strong enough?
  3. Is the code, re-usable where it makes sense?
  4. Are functions small and code, simple to read?

8. What metrics will determine that the feature has been implemented successfully?

Finally, after all the fun-time you had creating the feature, what will determine that the feature has been implemented successfully?

The data you tracked will be of paramount importance here.

It can be the case that to track this quantitatively is not possible, but can you track the qualitatively in that case?

The idea here is that you can’t improve what you can’t measure?

Processing 100,000 requests? Fewer errors by the users? 95% work done by the new system instead of old one?

This can and will involve more stakeholders of the team and not just the developer.

Conclusion

Obviously, this is not the exhaustive things to take into consideration while designing a system or a product as an engineer. This just covered what I have learned so far by just doing things wrong or not right enough multiple-times.

It’s fun to build stuff! Continuously improve (“Kaizen” in lean)! Keep iterating! Keep shipping!

 

 

That’s all, folks!

 

IIMBx: EP101x DO Your Venture : Course Experience on edX

The best way to win is to show grit and bid on your strengths and intuitions. I have been trying to start my own company now for quite a some time. This course came into my mail box by edX, title looked captivating and thus I decided to take it.

It was a light course. Not much effort needed other that just watching the videos and doing some assignments.

The course was focused on making people get off with their venture from the idea phase to execution phase.

Throughout the course I was evaluating  my ideas and operations using the learnings given by the course and it has been of some help at least.

The course was divided into 5 weeks :

Week 1 : The “Do” Philosophy 

DO philosophy is based on “do or do not. There is no try”. There is a gap between intent and action and thus best bid is to don’t wait for halcyons days and just DO IT.

It proposed a term equifinality which means that every entrepreneur and his path is different and there are many ways to entrepreneurship.

It then provided interviews with various startups like Bums On Saddle, Hobby in a Box etc founders sharing their journey so far.

Week 2 : Opportunities, Idea Creation & Generation!

This week was focused on how entrepreneurs come up with ideas. Few of them are  :

  • Hobby driven ideas
  • Painstorming
  • Change in some rules and regulations

Each of these points can provide you with a nice venture idea. I already had my idea prior to the course and thus just thought about it again from these view points.

Week 3 : Idea validation and Evaluation

After the idea has been selected. We have got to validate and evaluate it.

Some if the methods for this are :

  • Personal feasibility
  • Market feasibility
  • Customer feedback

Best way to validate idea is to talk to customers. Real unknown customers. Not just people in your acquaintance. We here at Rainbow Shelf went out to talk to retail shop owners (around 10) and tried to explain them the idea. Response was not very encouraging but it was an experience that helped us.

Week 4 : Lean Canvas

This week was all about the lean methodology used in startups. I have read the book The Lean Startup  and thus this all made sense to me.

The 5 principles of lean startup are :

  1. Entrepreneurs are everywhere
  2. Entrepreneurship is management
  3. Entrepreneurship is validated learning
  4. Build, Measure, Learn
  5. Innovation Accounting

The course also provided a lean canvas that should be used by startup to assess their idea.

Week 5 : Effectuation

Effectuation, which is defined as a “logic of thinking, discovered through scientific research, used by expert entrepreneurs to build successful ventures“.

This week was about measuring the uncertainty in the entrepreneurship.

Effectuation has these 4 principles :

Bird in Hand Principle – Start with your means. Don’t wait for the perfect opportunity. Start taking action, based on what you have readily available: who you are, what you know, and who you know.

Affordable Loss Principle – Set affordable loss. Evaluate opportunities based on whether the downside is acceptable, rather than on the attractiveness of the predicted upside.

Lemonade Principle – Leverage contingencies. Embrace surprises that arise from uncertain situations, remaining flexible rather than tethered to existing goals.

Crazy-Quilt Principle – Form partnerships. Form partnerships with people and organizations willing to make a real commitment to jointly creating the future–product, firm, market–with you. Don’t worry so much about competitive analyses and strategic planning.

 

Overall. I wasn’t expecting very complex and hard material in the course just normal guidelines stuff and this was exactly what the course provided. This course was offered by IIMB. And showed many startups that are incubated there are NSRCEL.

As of now I have taken many scattered courses on entrepreneurship from University of Maryland to MIT to IIMB. Now i have decided to just take the specialization offered by Coursera  and that should be enough for time being until my company scales.

I have now shifted my focus to learn more core technical and domain knowledge that will be required to build remarkable company that can stand out that bring value to its customers.

 

The Lean Startup : Reading Experience

 

My 3rd book of the year was The Lean Startup by Eric Ries. I first got to know about the existence of this book during a keynote video of Gary Vaynerchuk where it was up for display. Finally got the time to read it.

The book focuses on how the lean manufacturing using in Toyota can be used in startups as well. And it makes sense! The case studies to new terms defined all help you shape your mind to run your startup in a lean manner.

Part One : Vision

Start

Traditional management taught in business schools is just not what an entrepreneurial manager need. The uncertain market, the uncertain product requirements all needs to be taken care of. This can’t be done with classic managerial metrics. A startup needs new metrics to track itself.

Define

A startup is human institution designed to create new product or service under extreme uncertainty.

 

Validated Learning

Failure is over hyped in startup world. People are happy to fail and then masquerade it as learning. But are we actually learning in the process?

Validated learning is the metric that startup needs to track its progress that it is making by learning from failures. Validated learning focuses on making use of learning to make tangible progress.

Experiment 

Get into the market as quickly as possible. Bootstrap the product and hit the market. Get the feedback of the customer. Nobody wants to end up building something that nobody wants. Find it as soon as possible.

Part Two : Steer

Build-Measure-Learn Loops

Eric suggests that startup should continuously run Build-Measure-Learn loops within the organization. Use metrics like innovation accounting and learning milestones to track actionable metrics and not to dwell on vanity metrics.

Leap of Faiths

Leap of faiths are the assumptions you make as an entrepreneur that your business depends upon. Entrepreneurs should have foresight, ability and tools to discover which of their leap of faiths are working and which are not.

There are two major hypothesis a startup depends upon :

  1. Value Creation Hypothesis : How you are looking to give value to customers?
  2. Growth Hypothesis : How you think that your product will grow?

Minimum Viable Product : Test It Out

Only way to test your hypothesis and leap of faiths is to hit the market.

Best way to do so is to create an MVP, that consists of your core business features. Make sure that customer actually wants what you are building.

A very good example is Dropbox : Their MVP was a video showing how it will work. It was enough to let them know that there is need for there product in the market.

Concierge MVP  : It is testing MVP with selected customers that you take feedback from in exchange of VIP treatment and support.

Measure 

How do you know that your product is improving? Innovation Accounting is again a metric that allows you to do so. It involves 3 steps.

  1. MVP
  2. Learn – Lean towards working business model
  3. Pivot or Persevere

Cohort Analysis and Split Testing

How do you know which change in product is steering the change in customer behavior? Cohort analysis and split testing is used to test different versions of products with different customers at the same time. This allows us to test features and do innovation accounting properly.

Kanban or Capacity Constraints 

Kanban is an agile development methodology that doesn’t allow new features to be added in backlog until implemented features are validated.

Pivot or Persevere 

The most important decision for a startup is to persevere current approach or pivot.  Pivot is special kind of change designed to test new business hypothesis about the product.

Eric states 10 types of pivots in the book :

  1. Zoom in Pivot
  2. Zoom out Pivot
  3. Customer Segment Pivot
  4. Customer Need Pivot
  5. Platform Pivot
  6. Business Architecture Pivot
  7. Value Capture Pivot
  8. Engine of Growth Pivot
  9. Channel Pivot
  10. Technology Pivot

PART THREE : ACCELERATE

Batch

Eric focuses on the point that startups should now follow large batch production systems but instead work on small batches. This means shortest possible release cycles and always keeping customer involved.

Grow

Startups should focus on sustainable growth. A sustainable growth is when new customers are drive towards the product by the actions of previous customers.

Engine Of Growths

There are three engine of growths that can exist in a startup :

  1. Sticky Engine of Growth : Customers stick with the product for long term.
  2. Viral Engine if Growth : Customers spreading the name of the product as side effect of using the product.
  3. Paid Engine Of Growth : Promotions and stuff.

Paid engine of growth is only profitable when customer lifetime value is greater than cost per acquisition.

Building an Adaptive Organization

Startups should focus on building an adaptive organization. Don’t go too fast nor too slow, don’t get too structures neither lack any structure at all.

The 5 Whys

As used in Toyota. 5 Whys is asking 5 level of whys on every problem. This helps you to get to the root of the problem.

Beware this should not become game of 5 Blames where each team keeps blaming other.

Innovate

Startups should provide platform for their employees to innovate. Best way to do so is to create an innovation sandbox. New features are added within this sandbox and are tested on early adopters segment of customers. Take validated learning out of it and move forward.

Also key is to hold the internals of the startup accountable for their actions. This will increase sense of belonging-ness among the employees.

 

It was a great read. All stuff you read about makes perfect sense. All the problems the book states are real world problem and if you are in touch with startups are not new for you.

 

My next book is Founders At Work : Story of Startups’ Early Days. Excited to read this one.

 

 

 

Business in Boxers : 1 Month Into Running a StartUp

Why the Series? 

(Inspired by book : Business In Blue Jeans)

Hi! This is Priyank. This series is about me pen downing my start-up ride. Don’t know if it’s fast or slow, all I really know is I’m gonna enjoy the ride.

Why the name?

Boxers is what you’ll find me almost every time in and business is what’s always in my mind and thus the name “Business In Boxers”.

The Beginning

Last month on 15th Feb, me and my friend sat down seriously deciding on which business to start. The interesting thing was I did this with 3 friends. 3 awesome business ideas that are practical and scalable. Next, we 4 formed a team. A team with a vision to launch multiple businesses in a very short period of time.

Our first startup what we call as Pebbles Media deals with creating engaging applications for local business owners that can help them discover new customers in their locality. Cool idea that we all thought will get us many clients.

We approached 5 clients in different domains from food business to education to mechanical work. Two of them showed interest and we started working on their products.

15 days later one of our clients refused to pay the advance thus we dropped him. Other client wanted a custom product built for them at a very unreasonable price, again dropped him.

The Pivot

Witnessing all this, we all after 1 hour long discussion decided that it is the time for an early pivot. We never wanted to enter service based B2B sector.

The pivot that will need us to make products that will be used by multiple small businesses and their customers making us a product based company.

A pivot at this early stage raised doubts in our minds but I guess it was the red flag that we all managed to see and decided to pivot and not to run for quick money.

In Love With Business Books

lately I have been reading a lets of business books. Getting ideas and knowing how various aspects of a startup works. My favorite is The Lean Startup. A must read for everyone.

The Speed Breakers

No surprise here, first month has been filled with speed breakers. I was not expecting any smooth ride either. This month we have seen from clients saying no to all saying yes at the same time. Client giving us advance and then we turning him down because we wanted to pivot.

Wrap Up!

This month we dealt with real clients. Realized that we don’t want to go into service based sector.

For next month we are planning to get our MVP out in the market and test our leap of faiths. Too much action for next month. Let’s see what’s in the store next.

See you next time.

 

PS : It is fun to write blog when you are pretty certain that no one will read it 😀