The challenge of managing complex systems

Using Supply Chain planning & optimization tools and the curse of "averages"

At 2DA Analytics, we solve a network distribution problem relating to how barrels of crude oil and refined products are sequentially scheduled from origin to destination, taking into consideration supply, demand, logistics capacity, inventory, and storage capacity. 

This problem has to be optimized over time, distance, and value simultaneously and is subject to all kinds of disruption events. This activity is currently done manually using spreadsheets and there is a reason why those tasked with making scheduling decisions have wound up doing it this way. 

We thought we would do a two-part series on why previous attempts at solving this problem with re-purposed software have left the oil industry long on promises and short on results.  

In this part, we look at supply chain planning software. Attempts to use production planning and optimization software to solve this problem, despite having operations research foundations and mathematical rigor behind them, have been a failure. While these software solutions frame the solution correctly, they are so badly outmatched to the size of the problem that the core mathematical model itself becomes a liability. The best way to explain this is with an analogous complex system that hopefully everyone can relate to easily.

The map below shows how air traffic control over the United States is divided into zones, called Air Route Traffic Control Centers (ARTCC) with one control center that manages air traffic through that zone except for the TRACON airspace and local airport airspace within that zone.

Terminal radar approach control - TRACON - handles departing and approaching aircraft within its space.

Air traffic control tower (ATCT). An ATCT is located at every airport that has regularly scheduled flights. Towers handle all takeoff, landing, and ground traffic.      

Aircraft on scheduled flights are orchestrated from airport to sector within a zone and then handed off across zones through this network and follow defined rules on defined flight paths. Changes in the weather, flight emergencies, delays, equipment failures are all managed locally, within and then across these zones to ensure that the 5,000 commercial and military aircraft airborne every hour are moved smoothly from origin to destination and, most importantly, don't bump into each other.

While the Federal Aviation Authority (FAA) has oversight and view over the entire network, it does not manage individual aircraft or airport movements at a centralized level. The system is purposely designed to be robust in the face of the complex nature of operations. The FAA intervenes as required to help the zones manage disruptions and has final authority over the rules and ability to suspend services if needed.

This all happens in real-time with aircraft moving very high speeds and looked like this today:

In our analogy, the cost of being wrong is extremely high and measured in human lives.

Because response times are critical, even under normal operating conditions, the system is designed to keep decision making as close to the activities with the highest risk - for example, take-offs and landings are controlled at the local airfield. This is recognition of how complex systems subject to continuous disruption events are incredibly fragile and not suited to top-down, centralized planning tools. The magnitude and velocity of changes at the margins of the system have a cascading effect on any "plan." In addition to being mathematically challenging to execute, the costs to operate and lag in response to "continuous events" routinely overwhelm benefits of any precision or accuracy gained.

Now to extend analogy back to the distribution problem in oil, we add another twist - suppose there is a strong economic incentive to route airplanes to an airport. Let's say that right now, for every flight that lands in Chicago, the FAA pays passengers a bonus of $1,000 each and the airline a matching lump sum equivalent to what all the passengers received. In comparison, flights arriving in Ft. Wayne, Indiana, gets a passenger bonus of $250. Flights arriving in Milwaukee, Wisconsin, are charged a fee of $500 for every passenger disembarking. These bonuses change several times a day. This is the value component that we mentioned at the beginning and because the "market independently sets it", introduces a probabilistic dimension to the system. We'll park this for now and revisit it later. 

So to apply a supply chain planning tool to manage the tactical aspects of keeping this complex system balanced and re-balanced in the event of disruption requires capturing supply, demand, available pathways between and constraints for all three. The typical engine used is a CPLEX solver that sets up a matrix calculation to solve for an objective function. The objective function can be something to the effect of "Efficiently satisfy all demand subject to these forecast margins by demand source to minimize carried inventory, subject to the following constraints for a defined time frame."

Now, to the actual mechanics oil distribution. First, the "route network" for oil below, in this case, major refined fuels pipelines and major bulk terminals in the United States. Each pipeline has 100,000's of barrels (think of them as passengers in the air traffic example) moving through them with millions of barrels of inventory at terminal locations (think destination airports).

 A two-dimensional cross-sectional view of a company's refined fuel flows ("scheduled flights") across a specific part of the network looks like this. These data visualizations are generated with live forecast data out of BAYZYEN.

The left side represents inbound supply (equity barrels from the refinery or 3rd purchases), the flows on the Sankey chart are the cumulative barrels moved down a specific path - in this case, individual pipeline segments on which barrels travel. Width represents volume from one node (terminals represented by horizontal line segments) to the next before sold through various channels - shown on the right side as two marketing channels in aggregate.

Isolating a single terminal shows the diversity of inbound and outbound barrel movements. Terminal to terminal pipeline movements, terminal to bulk sales, terminal to truck rack, bulk purchases (done in terminal or off a pipeline).

There are 45 terminals, and 25 pipeline segments connecting each terminal back to the entry point of volume in this example.  Some terminals have two pipelines delivering and one receiving, others one line delivering and two lines receiving.  Not reflected here are pump-over lines between terminals, spurs to a trunk line, dedicated Jet A1 pipelines to airports. All of which have to be planned, scheduled, and managed individually, some of which are done entirely offline of any system of record - meaning there is no visible plan.

 What is shown above are volumes associated with one product/grade. Typically a terminal will have at least 6 unique product and their associated grade(s) simultaneously with pipeline, marine, rail and truck supplying all 6 products in a sequenced manner. At times, mandated grade shifts, seasonal fuel demand, re-grading from one product to another at the terminal - cause the number of unique products transported or held in stock to go up to 8-10, but let's stick with 6 for now. Each one of these products has a unique demand profile. The larger volume products have 3 to 4 demand profiles by a channel of trade. In terms of the chart above, there are at least six overlays representing each unique product and grade.

Next, the constraints need to be accounted for. Each terminal has product-level inventory minimums and maximums representing hard limits (in terms of storage capacity) which in this example equates to 540 constraints (6 products x 45 terminals x 2 capacity limits). Each pipeline segment has a capacity constraint relating to the maximum pump rate = 150 constraints (6 products x 25 pipeline segments) 

In terms of a centralized optimization solution, using a CPLEX based Linear Programming approach, this represents a decision matrix that seeks to maximize 10 demand profiles (3 demand profiles for 2 products and one profile for the remaining 4 products) per location across 45 locations, subject to 690 constraints (summing up all identified constraints) per day, going forward for 30 days. With the variability associated with demand, supply, and transportation capacity, this solution will need to be re-optimized at minimum daily, as conditions warrant. Not included in this optimization are the economic incentives tied to demand profiles, as we noted previously in the air traffic example. These change multiple times a day and thus can have a material impact on the starting inventory position for any optimization solution.

At 2DA, we have seen customers using this approach suffer excessive computation time to generate a feasible solution. At an international oil company, a Long Range Planning (LRP) tool used to optimize weekly transport schedules took more than 8 hours to compute, rendering a result that was obsolete by the time it was published. 

The natural tendency is to optimize computational run time, by reducing the objectives to a historical mean or aggregate components to higher-order classifications.  For example 9 and 11 RVP gasoline is aggregated to a generic "Sub-Grade" or "Premium" classification (or, further to generic product levels such as "Gasoline"), which defeats the whole purpose of computational precision and reduces scheduling to an activity easily and rapidly achieved using spreadsheets. 

If the objective function is to "have in stock, an average amount of Gasoline at this terminal over the next 7 days to meet a cumulative seven day average of aggregated Gasoline demand", it only takes a small change in any one of the inputs to cause a cascading effect in the "optimized plan". More critically, what is lost is the ability to optimize the incremental barrel against specific channel demand and economic margin.

Imagine boarding a flight to Chicago from New York and being told, ".. on average, the flight will land within three hours of the expected arrival time and, on average, within 200 miles of Chicago."  Every Commercial Leader - whether Supply, Trading, or Marketing - in an Oil Company knows exactly how this sounds at the end of the month when inventories, economic margins, and costs all miss forecasts. It is ironic that in the quest to gain precision, the use of this type of software results in optimizing to the mean.

At 2DA, we have always eschewed averages and our software is specifically designed to allow our customers to see the real topology of their asset networks so that they can efficiently maximize every incremental barrel in their system. A digital twin of their commercial infrastructure.