1/13/2024 0 Comments Iceberg hive![]() ^ a b "Incubation Status Template - Apache Incubator".^ "Initial public release in apache/iceberg".^ "Using Apache Iceberg tables – Amazon Athena"."Apache Iceberg: The Hub of an Emerging Data Service Ecosystem?". ^ a b c d e Woodie, Alex (8 February 2021).List of Apache Software Foundation projects.Iceberg is used by multiple companies including Airbnb, Apple, Expedia, LinkedIn, Adobe, Lyft, and many more. In May 2020, the Iceberg project graduated to become a top level Apache project. The project was open-sourced and donated to the Apache Software Foundation in November 2018. Simplify and obfuscate general operation and maintenance of tables.Improve performance by enabling finer-grained operations to be done at the file granularity for optimal writes.Ensure correctness of the data and support ACID transactions.Ryan Blue set out to address three issues that faced the Hive table by creating Iceberg: Many at Netflix avoided using these services and making changes to the data to avert unintended consequences from the Hive format. Hive was never able to guarantee correctness and did not provide stable atomic transactions. Hive was used by many different services and engines in the Netflix infrastructure. Iceberg started at Netflix by Ryan Blue and Dan Weeks. Vendors currently supporting Apache Iceberg tables in their products include CelerData, Cloudera, Dremio, IOMETE, Snowflake, Starburst, Tabular, and AWS. Iceberg addresses the performance and usability challenges of using Apache Hive tables in large and demanding data lake environments. Iceberg is released under the Apache License. Iceberg enables the use of SQL tables for big data, while making it possible for engines like Spark, Trino, Flink, Presto, Hive, Impala, StarRocks, Doris, and Pig to safely work with the same tables, at the same time. Apache Iceberg is an open source high-performance format for huge analytic tables.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |