Guides

Task-oriented walkthroughs for loading, transforming, and managing data in Magpie.

Importing and Exporting Data

• Accessing Data from MySQL
• Creating and Managing Data Sources
• Accessing Data from Distributed File Systems
• Accessing Data from JDBC Sources
• Accessing Data from Redshift
• Accessing Data from Snowflake
• Accessing Data from BigQuery
• Accessing Data from PostgreSQL
• Accessing Data from Kafka
• Accessing Data from Generic Sources
• Loading Data from URL
• Exporting Data from Magpie

Managing Jobs, Schedules, and Notifications

• Creating Jobs
• Scheduling Cluster Activity
• Scheduling and Executing Jobs
• Setting Up Notifications

Managing Clusters

• Creating and Altering Clusters
• Downloading Files onto Clusters
• Installing Python Libraries during Cluster Bootstrapping
• Starting and Stopping Clusters
• Configuring Cluster Health Notifications

Security

• User Management
• User Roles
• Magpie Permissions
• Activity Monitoring in Magpie

Supported Languages

• Using Python in Magpie
• Using Scala in Magpie

Advanced

• Cluster and Job Observability Commands
• Create Secret
• Drop Secret
• Notebook Revision Control
• Personal Access Tokens
• Introduction to the Spark Web UI
• Substituting Variables and Secrets at Runtime
• Connecting to BI Tools