92% of organisations experience production issues in low-code platforms each year due to inadequate testing. The finding comes from a report published by Copado entitled, “2022 State of Salesforce Testing Report”. Its findings are based on a survey of 275 executives, managers, and members of Salesforce development teams, mainly from larger organisations with more than 1,000 employees.
The report, a weighty 47 pages in length, could have been much shorter and is more in the form of a PowerPoint presentation than a whitepaper. Its overall theme highlights the importance of DevOps and testing for any Low-Code platform, but it focuses on the Salesforce platform.
The report has an executive summary and is divided into five sections:
- Common Salesforce testing challenges
- Testing trends
- Testing 101: Models, Types, Teams
- Testing Strategies and Solutions
- The Evolution of Testing
The second section is misnamed as it highlights the challenges and issues faced by testing teams. The evolution of testing section would be better named Testing Trends as it includes five trends Copado feels will impact the industry, these are:
- Artificial Intelligence and Machine Learning
- Robotic Testing
- Low-code Testing
- Robotic Process Automation
- Hyperautomation
The final section reveals the methodology used, which includes, oddly, the call to action for the report.
Pat McQueen, Senior Vice President of Growth at Copado, summarized the report saying: “Companies are clearly finding that automated testing is the best way to move fast and not break things. Teams that prioritize their testing strategy outperform those who don’t. The investment in automated testing enables business agility which leads to long-term commercial success in a competitive market.”
Some highlights
The authors recognise that Low Code platforms like Salesforce help organisations enormously with digital transformation. However, it is not a magic wand that suddenly creates a framework for transformation perfectly, as highlighted by the key finding above.
Companies need to consider how they introduce testing into their DevOps process. For 84%, this is still partially a manual process. However, this also means that 41% of teams do not have the time to test every change before it is released.
Challenges
The report looks at the complexities of testing the Salesforce low code platform, specifically noting three key challenges:
- Salesforce doesn’t exist in a vacuum, and changes impact other apps
- Salesforce apps require constant updates and feature dynamic content
- the Salesforce platform is highly configurable
Automated testing can alleviate the amount of manual testing required and ensure that fewer defects reach production. Most organisations recognise that testing is complex and needs people.
In 20% of organisations, more than 100 people are dedicated to Quality Assurance. 26% of respondents even outsource testing, which in itself has concerns. Another 23% stated that developers primarily do their testing. It would have been interesting to see which industries this was prevalent in, but the sample size was too small.
Should QA be part of IT or the business? Is testing just the responsibility of IT? Should QA be independent or just another part of IT? It is a question that has been around for decades. QA, sadly, is often the first team to be cut, mainly because its value is understated.
The Testing 101 section is a lightning-fast tour of testing and is worth a read, though it has no extracts from the survey that see what methodologies are in use. The Testing Strategies and Solutions section complements this with insights about the balance between manual and automated testing used by the respondents.
However, the rest of the section delivers a set of questions for firms to ask themselves. It lays out some useful information about testing strategies, including security testing. It does not highlight what organisations are doing, best practices rather than survey insights.
Benefits
The benefits of automated testing are there. The report found that those organisations with automated solutions have more success.
- 50% more frequent releases
- 58% fewer production failures per year
- 50% more likely to complete testing for each release
- 50% lower total cost of ownership
Also, the later in the process that defects are found, the more costly they are to the business. According to Jones, 2008, defects identified in production cost 640x more than those found in coding.
Enterprise Times: What does this mean
Whilst this report purports to indicate the state of Salesforce testing, the survey results provide only a very high-level overview of what people are doing. Future reports would add more value if the detail around testing were delved into. For example, how many are doing security testing and to what level?
However, the meat of the report around what testing a Low Code platform such as Salesforce should look like is worth reading. There are useful insights, checklists and questions for the reader to consider their testing approach.
The punchline that plays to Copado’s strengths as a DevOps and testing solution provider is that manual testing has a 95% higher total cost of ownership than using commercial test automation tools. The real financial numbers behind this statistic are hard to prove, as most are hypothetical, but the research by Caper Jones would indicate that it is valid if automated testing highlights flaws earlier in the development process.
In summary, Low Code platforms are not always perfect solutions to deliver digital transformation, but with automated testing, they can get a lot closer.
