From the course: Complete Guide to Databricks for Data Engineering
Join today to access over 24,600 courses taught by industry experts.
Notebook job scheduling - Databricks Tutorial
From the course: Complete Guide to Databricks for Data Engineering
Notebook job scheduling
- [Instructor] Notebooks job scheduling. So far, we have done lots of data cleaning and data processing, and we were manually executing this notebook. So the question come is how we can automate it? And that is what we can do by doing the job scheduling. Let's just understand what is notebooks job scheduling? Notebooks job scheduling is a mechanism or a feature through which you can automate your notebook's execution. So far we are manually executing this notebook cell by cell, but now with the notebook scheduling, you can decide when and how you want to execute this specific notebook automatically. It eventually help you to create, manage, and monitor your data pipelines in more smoother and efficient manner. If we talk about some of the important features of this notebook scheduling, it will be first, automatic execution. You can decide that, "Okay, I want to execute my pipeline at 12 o'clock midnight every day." Now in that case, you don't have to stay awake till 12 o'clock to…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.