From the course: Manage and Optimize Big Data with Apache Iceberg

Unlock the full course today

Join today to access over 24,600 courses taught by industry experts.

Setting up your Iceberg environment on Databricks

Setting up your Iceberg environment on Databricks - SQL Tutorial

From the course: Manage and Optimize Big Data with Apache Iceberg

Start my 1-month free trial Buy for my team

Setting up your Iceberg environment on Databricks

“

- [Instructor] Let's log into our Databricks account. I've already logged in. To use the Apache Iceberg, we need to install the Apache Iceberg JAR in our Databricks cluster. By default, Databricks uses the Delta Lake, so we need to install the Iceberg jar. Now to get to know which JAR you need, let's go to the Google and search Iceberg Spark compatibility. You will find a page where there are information given about which Spark version support which specific Iceberg version. So this is the compatibility metrics. Here you can check, based on your Spark version, you can download the specific Iceberg runtime JAR. Let's go back to my Databricks, go to my cluster, and let's see my cluster version. My cluster version is Apache Spark 3.3. Let's go down here and see for Spark 3.3, this is the specific Iceberg JAR. I can click on this JAR to download. Now this is started download. Once your JAR get download, under your cluster, click on the Libraries, and click on the Install New. Now here we…

Contents