I don't get where this 50% comes from? When I take the full list of the prices (https://aws.amazon.com/emr/pricing/), I get as low as 1,10% EMR cost for p3.16xlarge and the highest is 30,77% for g2.2xlarge.
There's a bunch of instance types that don't make sense in an EMR cluster, in my experience the EMR markup is rather ~10-15% on average, which seem rather OK for the service it offers.
Ok that makes more sense, thanks for clarifying. Still though, when I do the math assuming Spot market saves you 67% of the price, I get that for i3.16xlarge (the instance we use the most, with EMR fee $0,27):
- On-demand $4,9920: 5,41% EMR markup
- Spot: $1,6474: 16,39% EMR markup
Indeed it adds up, but it's rare to get an average of 67% discount all-the-time with no fallback on on-demand.
Anyway, I think the bottomline of your article is: most instance types do not make economical sense to use in EMR, pick wisely.
On top of choosing the instance type wisely, my other question is which proprietary software from AWS requires the per-instance pricing!!! In comparison with the EKS cluster, which requires only per cluster fee.
Indeed, sorry it feels I missed your point but I'm not. My rationale is that I get value for my money with AWS packaging coherent set of tools that just require me changing a variable to migrate, provided I use an appropriate instance type that contains the relative EMR markup in an acceptable bracket (10-15% in my case).
There's no license cost or whatever that would justify paying 50% markup.
I don't get where this 50% comes from? When I take the full list of the prices (https://aws.amazon.com/emr/pricing/), I get as low as 1,10% EMR cost for p3.16xlarge and the highest is 30,77% for g2.2xlarge.
There's a bunch of instance types that don't make sense in an EMR cluster, in my experience the EMR markup is rather ~10-15% on average, which seem rather OK for the service it offers.
50% surcharge coming from the assumption that EMR running on spot instances
Ok that makes more sense, thanks for clarifying. Still though, when I do the math assuming Spot market saves you 67% of the price, I get that for i3.16xlarge (the instance we use the most, with EMR fee $0,27):
- On-demand $4,9920: 5,41% EMR markup
- Spot: $1,6474: 16,39% EMR markup
Indeed it adds up, but it's rare to get an average of 67% discount all-the-time with no fallback on on-demand.
Anyway, I think the bottomline of your article is: most instance types do not make economical sense to use in EMR, pick wisely.
On top of choosing the instance type wisely, my other question is which proprietary software from AWS requires the per-instance pricing!!! In comparison with the EKS cluster, which requires only per cluster fee.
Indeed, sorry it feels I missed your point but I'm not. My rationale is that I get value for my money with AWS packaging coherent set of tools that just require me changing a variable to migrate, provided I use an appropriate instance type that contains the relative EMR markup in an acceptable bracket (10-15% in my case).
There's no license cost or whatever that would justify paying 50% markup.