HDFS Credentials

May 28, 2020 17:07

AWS Access Key Credentials

{
  "credentialType": "AwsAccessKey",
  "accessKeyId": "<string>",
  "secretAccessKey": "<string>"
}

Field Name	Type	Description	Required
credentialType	String	The type of credential. `AwsAccessKey` for AWS Access Keys. Only for use with `S3A` sources.	Required
accessKeyId	String	The access key ID part of the keys.	Required
secretAccessKey	String	The secret access key part of the keys.	Required

{
  "credentialType": "AwsAccessKey",
  "accessKeyId": " AKIAIOSFODNN7EXAMPLE",
  "secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}

{
  "credentialType": "AzureServicePrincipal",
  "tenantId": "<string>",
  "clientId": "<string>",
  "clientSecret": "<string>"
}

Field Name	Type	Description	Required	Default
credentialType	String	The type of credential. `AzureServicePrincipal` for authenticating using an Azure AD service principal. Available for use with `abfss` and `adl` type sources.	Required
tenantId	String	The tenant (directory) ID that target resource resides in.	Required
clientId	String	The client (application) ID of the Azure AD service principal. By default, Magpie uses the cluster’s default client ID and secret.	Optional	Magpie cluster client ID
clientSecret	String	The client secret of the Azure AD service principal. By default, Magpie uses the cluster’s default client ID and secret.	Optional	Magpie cluster client secret

{
  "credentialType": "AzureServicePrincipal",
  "tenantId": "fb74c09d-d7de-4f68-b826-35bc80cc288f"
}

{
  "credentialType": "AzureSasToken",
  "token": "<string>"
}

Field Name	Type	Description	Required	Default
credentialType	String	The type of credential. `AzureSasToken` for authenticating using a shared access signature token. Available for use with `abfss` and `wasbs` type sources.	Required
token	String	A generated SAS token for the storage account or container.	Required

{
  "credentialType": "AzureSasToken",
  "token": "?sv=2018-03-28&sr=c&si=myPolicyPS&sig=NAxefF%2BrR2ubjZtyUtuAvLQgt%2FJIN5aHJMj6OsDwyy4%3D"
}

{
  "credentialType": "AzureSharedKey",
  "key": "<string>"
}

Field Name	Type	Description	Required	Default
credentialType	String	The type of credential. `AzureSharedKey` for authenticating using a storage account access key. Available for use with `abfss` and `wasbs` type sources.	Required
key	String	The access key for the storage account.	Required

{
  "credentialType": "AzureSharedKey",
  "key": "lLrCcuiWVsLCoEXAMPLEBjs/ivkLEEXAMPLESU3Nu27ybphEXAMPLEmwjMwMHoHxEXAMPLE/UhLowWQ=="
}