The launch of Carolina Cloud’s custom Nextflow executor is around the corner. In this post I share some internal details to convince readers they should at least try it out, and help name it.
Don’t have time to read the full article and just want to try it out? No problem, paste this into your LLM
Please read https://console.carolinacloud.io/static/nextflow/local-quickstart.md
and help me get started running a test pipeline (nf-core/rnaseq -profile test)
and then running a pipeline of my own.
But if you actually enjoy reading about this stuff then…
The launch of Carolina Cloud’s custom Nextflow executor is around the corner. In this post I share the hard numbers on speed up against the competition.
Most important chart in this entire article
tl;dr We benchmarked nf-ccloud vs AWS Healthomics and AWS Batch (via Seqera UI, both on-demand and spot). nf-ccloud finished in 4.5hrs at less than $20. AWS Healthomics finished in 5.5hrs at $45. Batch on-demand ran around 7-8h at $70 and batch spot ran anywhere between 10h and 24h but was the cheapest ad just under $10. You can view the Carolina Cloud run here. Shout out to Flock Bio for helping us benchmark.
A case study in hybrid CPU-GPU optimization and seamless container resizing.
Apexomic
The worst month of the year in the Northern Hemisphere is no doubt February. But this February we got some technical vitamin D as Carolina Cloud collaborated on a project with Apexomic, a bioinformatics consulting company based in Stevenage UK. They offer bespoke data analytics for the life sciences sector. They bring more than 12 years of experience to the table with expertise in both biology and business. Derek and I felt a connection back to our East Anglian past (where it all began in a number of ways).
After many requests, this week Carolina Cloud launched its own s3-compatible storage. We are starting with 100G free to all users with the ability to scale up rapidly if needed. Coupled with our no egress fee policy, this means our storage is effectively and in practice free! Why free? When we were deliberating pricing, in order to maintain our ‘at most 1/3 AWS’ value, we would make such little money on storage that it wasn’t worth charging.
When we first launched Carolina Cloud we had a pretty simple console frontend. We started with just VMs and containers so all we needed was four plain html files. Doing two of everything; creation modal, instance list etc was not too much hassle to understand or maintain. And besides, Cursor whipped it up for us and it worked first time!
But as the company, software and hardware was progressing so had to the frontend. We began doing what many early-stage teams are doing right now — we ‘vibe coded’ and to quote myself ‘I don’t care how the frontend is coded. If it looks good, it is good’.
A genomics startup with less than a year of runway needed to build a pipeline for processing cattle genomic data—merging chromosomal data from hundreds of animals to identify genetic markers. They’d been using AWS HealthOmics for other genomics tasks, but as they evaluated their options for this greenfield project, two problems emerged: the cost would be prohibitive, and the developer experience would make iteration nearly impossible.
This is the story of how they built their pipeline on Carolina Cloud’s bare metal Threadripper infrastructure and gained the development velocity they desperately needed—with 70% cost savings as a bonus.
After six months with Azure OpenAI’s Whisper API, a major call center needed a better solution. They were processing hundreds of thousands of minutes of audio per day, and costs were climbing. More importantly, they needed speaker diarization—something the OpenAI API simply didn’t offer.
Without diarization, they couldn’t answer critical performance management questions: What percentage of agents proactively offered a specific solution? What percentage mentioned key product features? Which agents needed coaching on particular topics? These insights were locked behind the technical limitation of not knowing who said what.
