Mapgie provides several mechanisms to export data. Typical use cases for downloading from Magpie include creating spreadsheets to share with non-technical audiences. Exporting data to another data source, on the other hand, can be useful if you want to make bulk copies of your Magpie data to another environment.
Export Result
As you may remember, the result of a Magpie command is accessible as result; using commands such as show 100 from result. Using a Magpie result, you can can export data from your environment by downloading results through your web browser or saving results to a data source (such as Amazon S3).
The export result command converts the dataframe resulting from an operation (usually a SQL query) into a file format of your choosing. The file is made available for download directly from the notebook. Only you are able to download results that you export. If another user wishes to download the same data, they must run the paragraph themselves. Example command:
sql(
select * from washington_dc.building_v_crime_dc
);
export result;
The default file format is csv. Also note that the default and maximum export limit is 1,000,000 rows.
See the full documentation for details: Export Result
Saving a Result to a Data Source
Instead of downloading a result, data can be exported from your Magpie environment to another data source, such as S3 or GCS. Command documentation: Save Result. Example command:
sql(
select * from washington_dc.building_v_crime_dc
);
save result as table myS3_building_v_crime_dc with data source myS3;
Saving a Table to a Data Source
In our last example, we selected all of the data from a Magpie table and saved that result to a data source. Since we were exporting all of the data from a single table, the Save Table As command is another, more succinct option for exporting our data. The syntax is very similar to Save Result, but it operates on a Magpie table rather than the last query result. Example command:
save table washington_dc.building_v_crime_dc as table myS3_building_v_crime_dc with data source myS3;
Security
To save data to a data source, you you may need to adjust the security configuration of your cloud environment. In particular, you will need to allow write access or object creation permissions for Magpie. Please reach out to a member of the Silectis team with any questions or support requests.