Project Aim
The test and test_full profiles in nf-core provide excellent tooling for testing pipelines before production deployment. However, these profiles currently over-allocate resources by default, leading to wasted compute and unnecessary costs during CI/CD runs and performance testing.
This project builds on previous optimization work documented at: https://github.com/FloWuenne/megatest-resource-optimization
Goals
Since this project aims at optimizing all nf-core pipeline test profiles, it is probably more suited for experienced Nextflow / nf-core contributors.
- Develop automated tooling to generate optimized resource configurations for each nf-core pipeline (Difficulty: Medium)
- Integrate this automation with existing CI/CD workflows and AWS Megatests (Difficulty: Medium/Hard)
- Create a system that automatically applies / adds optimized configurations after successful AWS Megatest runs (Difficulty: Hard)
The end result will be more efficient test profiles that reduce computational overhead while maintaining testing reliability.
