From the course: Using Large Datasets with pandas
Join today to access over 24,600 courses taught by industry experts.
Big data is dead - pandas Tutorial
From the course: Using Large Datasets with pandas
Big data is dead
- [Instructor] So a lot of companies are fearing that they have a big data issue, but I'm here to say that maybe it's not an issue for you. There is a great article called "Big Data is Dead" by the company who makes Duxdb, and they have several points there. One point is that when you look at companies and how much data they store, it's usually in the hundreds of megabytes, not that much more, right? Your company might be different, but a lot of company, that's the median size of data. Another point is that you usually don't touch all of the data. When I'm doing calculation for billing, I don't need to touch the address. Maybe just when I'm sending the bills, but when I'm calculating, I don't need to do that. Another issue is that data is a liability, which means that if you can manage to store less data, you're going to pay less for storage, less for running calculations, and there is a liability of if you store…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.