Mainfreight - Application Landscape project
THE MAINFREIGHT CHALLENGE
“Special people, special company” embodies Mainfreight and its unique culture. With a 1000+ strong global team, the Mainfreight team will tell you it’s like being part of one big blue-blooded family. Their DNA is carved out of hard work, ingenuity, and a deep care for one another.
The Mainfreight warehouse and transport teams had acknowledged that their testing was predominantly manual. There were limited opportunities to scale. Quality and consistency were predominantly reliant on individuals. Test costs and elapsed times were impacting on business profitability, time to market for new solutions and changes to products and services.
OUR APPROACH – THE ENHANCE PLAYBOOK
The Enhance playbook has four pillars – Capture, Curate, Transform and Release. Their detailed application to Mainfreight’s business on the Application Landscape Project was as follows.
Capture:
Here the magic begins. We start by listening to and understanding your current test environment and testing process and its alignment to your business strategies and priorities. We then scope your requirements, business needs, challenges, and opportunities. We combine these into our spell book to pull together a tailored automation strategy, playbook and approach specifically focused on the areas where you will obtain maximum business value.
Enhance conducted a 5-day investigation (Software Quality Assessment) to capture the state of the Mainfreight application and testing landscape. We investigated their manual and automated testing processes and curated a summary of the current testing practices, before providing recommendations to help transform the Mainfreight testing landscape.
Enhance was able to gain a detailed understanding of Mainfreight systems including business priority/criticality, fragility and quality of the systems including Mainfreight’s known and unknown risk appetite, and insight into the elapsed time, overall effort, and cost of testing.
There was a reasonably good level of consistency across the testing practices of the different application teams. What was reported back or uncovered by Enhance were challenges that are common to many organisations:
Testing was managed through Spreadsheets. Significant time and resource were spent on maintenance and reporting utilising inefficient reporting strategies, consequently providing a lack of transparency and timeliness of reporting.
Given the complexity of end-to-end transaction flows, test data was created manually and was time consuming. Focused automation would save considerable time and cost, replacing the high frequency manual tests and costly release smoke tests. Automation would also provide personnel with additional capacity to improve the overall quality of releases.
With globally dispersed teams, testing wasn’t consistent due to different interpretations or variations on what needed to be tested. Again, automation provides for consistency of what is tested, and the outcomes achieved.
Curate:
We curate a vision and develop a roadmap for transforming the state of software test automation within your business. As specialists, we can remove the pain points with our innovative approach, test automation tools, and talented resources.
Once Enhance had completed an in-depth Software Quality Assessment, using online and in person interviews to understand the good (‘Blue’ at Mainfreight), the bad and ugly. Our recommendations were provided at a general, application and strategic level.
Within the initial 5 days, the Enhance team used their findings to develop and demonstrate to the Mainfreight team operational and strategic recommendations and some automated working Proof of Value testing.
Enhance began to curate a roadmap for change for Mainfreight, that included the creation of a sample project using CucumberStudio, converting several high value manual test cases from across different applications to use as an example of how Mainfreight could get the most business value from test automation.
Behaviour Driven Development (BDD) test design was introduced and implemented to convert all test cases for each application to show how BDD / Cucumber may be used to define tests in a clear and self-documenting way that integrates with the automation code.
Enhance worked alongside Mainfreight performing these services and ensuring that the services and outcomes were as discussed and agreed. The Enhance Magicians to support this work, consisted of a Test Practice Lead and two Test Automation Engineers. Enhance worked in unity with the Mainfreight team members and focused on implementing the following deliverables over a 12-month period:
Test case management tool set up in Jira / Xray
Test automation – Smoke and Regression test suites covering both the Sandfield and the Theta applications for the Warehouse and Transport teams.
Improved test visualization / reporting in a single location to ensure better understanding of the test coverage and to make test reporting easier
Transform:
Our magic is woven into our customers’ DNA – by automating tests, we can create new efficient operating methods that provide a transformed way of working with a strong focus on ROI.
Test Case Management Tool
To alleviate the heavy workload and dependency on spreadsheets for testing and test case management, Xray was chosen as the test case management tool to be implemented. Enhance guided and mentored the Mainfreight team on how to use it and move to replacing their existing / tools and processes, as well as change their processes to be more agile.
Automation Project
A Mainfreight-driven automation project was kicked off across transport and warehousing applications to deliver enhanced reporting practices. This was achieved via continuous liaison with the Mainfreight business users and vendors to scope the immediate problems to be solved with automation. We also formulated a long-term strategy across Mainfreight’s entire logistics business.
Enhanced Reporting Practices
Jenkins boards were created, and real-time test results were displayed on large monitors in offices. Supporting the Mainfreight brand and promoting a quality culture, Mainfreight blue is used to depict all positive test results.
Enhance Capability Embedded in the Mainfreight Team
Enhance Automation Engineers were imbedded with the Mainfreight team to translate the business needs and test requirements into automated tests that provide the greatest business value to Mainfreight.
Release:
We quickly, yet precisely, execute, deliver, and maintain what we have promised. Our robust and scalable operating systems, solutions and products wow our customers, demonstrating that we are the best in our field.
Mainfreight Benefits - Test Automation
We have been focused – with the Mainfreight team and suppliers – on reducing test elapsed time by one week per release cycle and per application. In support of increasing Quality Maturity, we are evolving the metrics and measures together with Mainfreight and their suppliers.
Benefits to date include:
Up to five times more defects found in regression testing using automation. Less time and cost to fix defects. Production failures minimised and brand reputation maintained.
Transport and Warehousing smoke test automations are saving an average 24 hours per release.
Reporting has become easier with Xray replacing google sheets, saving an average 2 hours per week which helps the team send out update/progress emails faster and frees them up to focus on other areas.
An Automated Regression suite has reduced the effort from an average 170 hours to 2 hours per release for a team of 17.
Mainfreight team can quickly identify issues after a major release and bring it up with the relevant development team. This works well for new team members learning regression tasks.
The time saved due to the automated tests has helped the teams; be available for daily support, increasing test coverage in other areas of the application and on other initiatives.
Mainfreight Benefits - Test Visualisation
While we had individual reports for the manual and automated tests, the Mainfreight team felt that it would be easier if there was a single dashboard that visualised both the manual and the automated test results, which will then make the understanding of the test coverage and test reporting easier.
Enhance developed and created a dashboard that actively integrates with Jenkins and Xray to fetch the latest results. This brings bring the manual and automated test results in one central reporting location. Benefits include:
Provides a single dashboard allowing Mainfreight to view the overall test progress achieved across the manual and automated test cases
Drives improved collaboration, transparency, and accountability across teams
Teams across NZ and AU regions are both able to utilise and benefit from the increase visibility these dashboards afford
Mainfreight Benefits – Quality Maturity
Enhance uses a four-stage maturity curve. The initial Software Quality Assessment indicated that Mainfreight test automation was bordering ‘Exploration’ when we began our engagement. As a result of this programme of work, Enhance and the Mainfreight team matured the Mainfreight testing environment to the ‘Transformation’ state of maturity. Benefits resulting from the increased state of maturity include:
Decrease in IP loss
A clear strategy/roadmap for the team to work to
Reduced manual effort frees personnel for higher value tasks
Comparatively shorter testing cycle and overall efficiency gains
Quality Assurance (QA) does not stand still, and like a Mainfreight truck or van, we just keep loading it up and delivering a quality result for Mainfreight, its software partners and its customers.
Mainfreight continues to grow and evolve its testing maturity. Through a partnership with Enhance, an enterprise-wide global strategic approach is being progressed. This will ensure that the rapid growth, momentum and change over the last 18 months is continued. This next stage progresses from Test Automation to end-to-end QA. This enterprise view will enable Mainfreight to hold themselves, and their current and future software partners accountable for high-quality software delivery in the following ways:
Consistency in delivery and expectations
Shared QA culture with continuous improvement
QA is integrated as part of the full lifecycle
QA and its effectiveness and costs are monitored
QA is continually fine-tuned
Process reuse is encouraged, practiced, and evolved