MES Software Testing – the What, the Why and the How

In September of 1999, 10 months into its journey to Mars, NASA’s Mars Climate Orbiter – a 638-kilogram robotic space probe designed to study Martian climate and atmosphere – unexpectedly burned and broke into pieces when attempting to approach Mars’ orbit. An investigation into the underlying reasons for the catastrophe revealed that the US-based company that designed and built the spacecraft, which also provided the crucial ground software for it, did so using the traditional imperial system of units instead of the metric units used all across the NASA. Not caught and remedied in time, the missing conversion of pound-seconds squared to newton-seconds squared resulted in an acceleration curve that was wildly different from predicted, which sealed Mars Climate Orbiter’s fate. This error cost NASA around USD 125 bn.

Make MES, not a MESS

A well-thought-out testing process can save businesses from potentially costly mistakes like the one made by NASA (though hopefully less spectacular). Testers/QA specialists are the extra pair of eyes trained to spot issues with your solution that would not occur to e.g., a developer or even a product owner. They also try to imitate the actions your users will perform in your MES system daily – that is why it’s so crucial for testers to know your requirements as well as your business operations. And, since it has been proven time and again that bugs become exponentially more laborious and more expensive to fix the later they are detected, it is wise to include testers in the development process as early as possible. As the folk saying would have it, ‘a stitch in time saves nine’ – testers can ensure that a minor issue does not become a showstopper once the solution is live on your shop floor.

‘I spy with my tester’s eye…

Software testing can roughly be divided into functional and non-functional. Functional testing focuses on answering the essential question: “Does the solution work like it’s supposed to,” looking for the answers from the smallest modules or operations to complex and layered integrations with external systems. Non-functional testing, as the name already suggests, concentrates on other aspects of the solution, like ‘is it easy enough to use,’ ‘will it handle peak times,’ or ‘is it easily scalable.’ Depending on the MES system and business requirements, either type of testing can be done in one of two modes, manual or automated – which are more alike than most people believe.

In manual testing, a human tester will meticulously check your solution hands-on, looking for any gaps, inconsistencies, and errors – screen by screen, operation by operation. They will do their best to try to break the system in various ways, to make sure that your users won’t be able to in a real-life situation. Manual testing is essential in verifying if critical operations and business flows work as designed – a tester is often the first to point out a flaw or inconsistency in the solution before it becomes a stumbling block later in the development. What is more, since testers assume the end user’s point of view, they will also spot any concerns with the solution’s user interface and user experience. After all, you need a solution that is easy to use and, ideally, easy on the eyes, too – and this is something only a human tester will catch.

To boldly go where man doesn’t need to go

The testing process does not stop once your solution goes live – tests should be run after each change, upgrade or extension of your MES system, to ensure that the unchanged functionalities still work as intended. Hiring a manual tester to repeat the same set of tests every several weeks or months is hardly efficient – this is where automated testing comes into play. A carefully developed automated test suite can perform the same actions a human would in a fraction of the time and with the same or better accuracy. Automated testing tools are also indispensable in performance tests – these focus not on detecting bugs but on checking e.g., how quickly your solution works, how stable it is, or how many concurrent users it can handle. It is highly impractical or downright impossible to test these aspects manually – even if hiring e.g., 100 testers, was an option, you could not get them to work in perfect synchronization. The tools can also quickly generate test data that would take a tester a lot longer to prepare. There is a wide variety of automated testing software available, from free demo versions (that are still quite sufficient for commercial uses) to paid licenses. They offer a vast array of functionalities and options – all that is left is choosing the best tool for the job.

Strike a balance between human and machine

You might ask at this point, ‘OK, if automated testing is so efficient and effective, why not automate all testing?’ The answer is simple: it doesn’t pay off. For some features or functionalities, writing and implementing a dedicated automated test script would take longer than just checking them manually, and some features may only need testing once. Automated test suites and testing tools need to be regularly updated by testers to reflect changes in your solution or business – it is essential to carefully select the scope for automated testing to make the maintenance as time- and cost-efficient as possible. The ratio of manual to automated testing will vary depending on the specifics of your business and your MES system – a well-designed testing process will balance the two to make the best use of your resources.

About the author

Zofia Wąchocka is a Software Tester at Andea, who has worked for several US customers specializing in composite materials manufacturing. A graduate of the Jagiellonian University in Kraków, her analytical approach to life and a drive for communication and comprehension brought her over to the tech world, where she thrives asking questions, sharing knowledge in webinars and articles, and learning test automation.

Content provided by

Łukasz Howorski is a Project Excellence Department Manager and a QA Architect with rich and diverse experience in test management, execution, and test automation. ISTQB and AGILE certificates help him create full test strategies that are tailored to client requirements. His technical and organizational skills allow him to implement frameworks not only in the test automation field but also in improving test processes using different approaches. He likes sharing his findings with the team, improving their skills one nugget of knowledge at a time.