Can't help responding since I see XTable, format interop vs unification there. ;). Great topic, needs more open debate in the industry, than big players forcing a standard with market presence.
On the topic of : "simpler to unify on one" - it feels like wishful thinking to me, from thought leaders. Engineers IMO enjoy their choices for whatever reasons they picked one project over the other for.
And is it even an achievable goal? I mean, even as we debate unifying 2 or 3, there's 2 more "table formats" claiming to optimize for ML or Streaming with a different storage layout/file format. Per computer science, this unification is as feasible as unifying all databases in the world.
Same vendors who want to unify on table formats, won't unify on the catalogs or even file formats anyway - so what's the point on unifying on table formats alone.
Interoperability : making layers of software work across these differences, in a way that's oblivious to the users is the progressive way forward. I feel this topic will not have this level of interest - without the high levels vendor interest/politics in this scenario.
Can't help responding since I see XTable, format interop vs unification there. ;). Great topic, needs more open debate in the industry, than big players forcing a standard with market presence.
On the topic of : "simpler to unify on one" - it feels like wishful thinking to me, from thought leaders. Engineers IMO enjoy their choices for whatever reasons they picked one project over the other for.
I sincerely don't know how to reconcile these technical differences. https://hudi.apache.org/blog/2025/03/05/hudi-21-unique-differentiators/ . Do we tell the world to slow down pipelines and spend more money while we figure out a standard?
And is it even an achievable goal? I mean, even as we debate unifying 2 or 3, there's 2 more "table formats" claiming to optimize for ML or Streaming with a different storage layout/file format. Per computer science, this unification is as feasible as unifying all databases in the world.
Same vendors who want to unify on table formats, won't unify on the catalogs or even file formats anyway - so what's the point on unifying on table formats alone.
Interoperability : making layers of software work across these differences, in a way that's oblivious to the users is the progressive way forward. I feel this topic will not have this level of interest - without the high levels vendor interest/politics in this scenario.