AWS Lambda loves charging for idle time: Vercel claims it found a way to dodge the bill Vercel claims it's slashed AWS Lambda costs by up to 95 percent by reusing idle instances that would otherwise rack up charges while waiting on slow external services like LLMs or databases. For the uninitiated, AWS Lambda is Amazon's serverless compute platform handy for short bursts of work, but costly for long-running or …
You've invented Kubernetes -- where nodes can last no more than 15 minutes.
Cool, fun, interesting - but when you know you're going to have many, MANY events, many connections, or persisted connections, why not use the proper tool for the job? An api gateway passing to a kubernetes cluster to schedule the work.
Threads, and Fibers, are such an old thing -- completely forgotten, and being reinvented anew. (Not yet, they're still forgotten - in favor of Yarn and Rope and and SpiderWeb and MultiLayerKevlar.) This -- using a fixed-life, expensive compute instance, individually, to handle long-lived workflows (handling multiple requests) by sequencing and process management - is just... why. These systems already exist, and work with systems that don't simply disappear out from under you. You're beyond the point of lambda, and you need to move to shared infrastructure of your own "management" (automated management - api gateways or load balancers to spread and dispurse calls, auto-scale instance groups or kubernetes to assign work load, etc.)
Yep, bypassing the lambda invocation mechanism and replacing the aws request dispatch loop in the lambda with their own does seem like reinventing the wheel. Add in the rest of the invocation stack, load balancers, client apis etc and you've got a beast of a hack that just seems like it's been done - better - before. I.e. using an established set of VM tools like k8.
Really it comes down to the aws lambda architecture decision to disallow concurrent requests. Only upside I can see is that if this gains traction aws might possibly revisit that decision. Though not holding my breath as all that paid for idle time serves their revenue stream well and allows them to pack more vms onto their bare metal
Agreed on the horrible hack to reimplement a capability best done properly elsewhere.
As someone who suffers in the day job from the lambda instance inflation under load and the associated cold starts I can kinda see where they started from, if not support where they ended up. Sometimes when in a hole, you keep digging.
I see the undoubted timesharing element in the narrow terms of getting the lambda to service multiple requests concurrently, something aws disallows natively. In terms of cost, pay aws once for the long dummy request keeping the lambda 'alive', and use it more efficiently by servicing other requests where otherwise the single task model would be stuck in an await on a dB request or whatever. Fewer cold starts when things get spicy as well for the win. It's a clever subversion of aws lambda's processing model, if unwise as a production architecture.
Can see how this would come up in a dev meeting as a cool idea, without stepping back and rethinking whether the whole lambda thing was a good fit in the first place.
I'm with you until the last phrase. Lambda is almost certainly a great fit for a lot of new services, as few new services (especially at startups) start out with 1>qps loads. The "in the first place" kicks in when your service actually starts being successful. In other words, Lambda is great for prototyping.
Yep, that fits us - we're not too big and lambda suits our modest use cases just fine, cold starts latency and all. As you say, quick to setup and good for unchallenging needs like ours.
I meant /their/ use case at the point where they embarked on this. Irrespective of how good a fit initially, if you're now incurring lambda usage costs significant enough to seek this sort of drastic engineering work-around, maybe you're at the point to step back and reconsider your tech stack, as other commentators have said.
Re engineering lambda into a container management plane and implementing your own replacement invocation stack isn't where I'd have gone next. I do understand though, sometimes you just keep digging.
> I moved ~20 client-side requests to server side (on 1 page). Now my Vercel bill jumped from $300 a month to $3,550 this month, where 99% of it comes from 'serverless function duration.'" The functions, he said, were "GET request to the DB and sending claude/anthropic requests"
Do these people not realize that their $3600/mo bill can be handled by *one* virtual instance at a cost of $30/mo? Seriously, a small instance, with the back-end of your choice (nginx + php? node.js? ruby on rails? what is new these days?) will *easily* handle 20 requests per second. In fact, you could get 200 to 2000 requests per second given the described workload, at $30-40/mo!
Every new start-up's first concern seems to be "We have to SCALE! Our 20 users will be 200 000 but this time next year!!" and they let it kill them by spending 100x as much as they ought to be spending, with no 200 000 user backing.
Assuming you don't count the cost of server administration, and don't count the cost of re-architecting for the planned transition to cloudy services later.
And of course this particular source of requests might only be 20 requests/second, but the overall application might be several orders of magnitude larger than that. I rather think you're right given that it looks like the entire bill was 3k
