[[Software testing MOC|Testing]] is frequently cited as one of the tougher challenges for developers adopting serverless. ## Understanding the challenges with testing serverless apps - [[Testing serverless applications is different from traditional software applications]] - [[Testing trade-off triangle]] - [[Pains with testing serverless applications]] ## Techniques [[Testing PRs with ephemeral stacks]] ## #OpenQuestions - What categories of automated testing can be applied to serverless applications? (unit, integration, e2e, etc) (see [[Testing honeycomb]]) - What are the [[Failure modes of serverless workloads]] to account for? - WHEN within the developer workflow or automated delivery pipeline is each category of test executed? - What are the different schools of thought currently within the serverless community about how to implement a serverless testing strategy? - Specific use cases where people ask: [[How do I test X in AWS serverless]]? - [[How do I prevent unwanted side effects when running E2E tests]] - Where would [[Smoke testing]] fit in a serverless testing strategy? Read-only automated tests against a deployed environment. - [[How to wait for an async task to complete inside an E2E test]] - [[How to verify publishing of message to an SQS queue when other Lambdas are listening to the queue]] - For E2E tests that invoke Lambda functions (either directly or via an async trigger), if they fail, is there a way for them to fetch the CloudWatch logs from the Lambda function and supply it in the Jest output? This would help with spped of resolution of failed E2E tests, where the cause isn't usually evident until developer manually goes and checks the CloudWatch logs. ## Training ![[Serverless courses and workshops#Testing]] ## Adjacent areas to testing - [[Monitoring serverless applications]] --- ## References - https://en.wikipedia.org/wiki/Software_testing#cite_note-Kaner_1-1 - [It does not run on my machine](https://coderbyheart.com/it-does-not-run-on-my-machine/) - Talk on testing cloud-native solutions by Markus Tacker - [How to test Serverless Applications](https://epsagon.com/development/how-to-test-serverless-apps/) by [[@Yan Cui]] - [Testing in production: Yes, you can (and should)](https://opensource.com/article/17/8/testing-production) by [[@Charity Majors]] - [Testing in Production, the safe way](https://medium.com/@copyconstruct/testing-in-production-the-safe-way-18ca102d0ef1) by Cindy Sridharan (41 min essay) - [Pains Testing Serverless Applications](https://serverlessfirst.com/pains-testing-serverless/) by me - [Falsehoods about Testing](https://club.ministryoftesting.com/t/falsehoods-testers-believe/1371) - [The best ways to test your serverless applications (Medium)](https://medium.com/free-code-camp/the-best-ways-to-test-your-serverless-applications-40b88d6ee31e) by [[@Slobodan Stojanović]] - [List of (mainly academic) resources on testing distributed systems (Github curation)](https://github.com/asatarin/testing-distributed-systems) - https://www.distributedsystemscourse.com - Ben Kehoe and other experienced serverless folks [talking about testing on Twitter](https://twitter.com/ben11kehoe/status/1296446757199544320?s=20). 2 schools of thought: 1 for cloud-based testing, 1 for trying to get local emulators for speed. - [The Devops Testing T](https://ingianni.eu/blog/devops_testing_t/) by Luca Ingianni: > By the way, if you look at it, this is really an embodiment of Gene Kim’s 3 ways: > systems thinking, in that QA is now an activity that encompasses the entire development and production system > amplifying feedback loops, in that you get many of them in many different places and phases of development and operation > continuous improvement and learning, as shown by the fact that these feedback loops never stop – you even receive feedback from production, potentially forever - [What should the scope of a serverless integration test be? (tweet thread)](https://twitter.com/donkersgood/status/1450121967596675079?s=12) by [[@Luc van Donkersgoed]] --- tags: #Serverless, #SoftwareTesting, #SoftwareEngineering, #MOC