What is Docker

What is docker? — IBM definition Docker is an open source containerization platform. It enables developers to package applications into containers—standardized executable components combining application source code with the operating system (OS) libraries and dependencies required to run that code in any environment Containerization — IBM definition Containerization is the packaging of software code with just the operating system (OS) libraries and dependencies required to run the code to create a single lightweight executable—called a container—that runs consistently on any infrastructure ...

June 9, 2022 · 3 min · Satvik Jadhav

Use PySpark Locally With Docker

Installing Spark on Linux Here we’ll learn how to install Spark 3.0.3 for Linux. We tested it on Ubuntu 20.04 (also WSL), but it should work for other Linux distros as well Installing Java Download OpenJDK 11 or Oracle JDK 11 (It’s important that the version is 11 - spark requires 8 or 11) We’ll use OpenJDK Download it (e.g. to ~/spark): 1 wget https://download.java.net/java/GA/jdk11/9/GPL/openjdk-11.0.2_linux-x64_bin.tar.gz Unpack it: ...

June 1, 2022 · 2 min · Satvik Jadhav