Awsflow Glue

Manage Glue ETL jobs, triggers, crawlers, and query the Data Catalog.

When to Use This Skill

Use this skill when the user:

• •Asks about Glue ETL jobs, crawlers, or triggers
• •Wants to inspect or query the Glue Data Catalog (databases, tables, partitions)
• •Needs to start a Glue job run
• •Wants to create a Glue job
• •Asks about Glue connections or job bookmarks
• •Needs to inspect crawl history

Tool: GlueTool

Execute AWS Glue commands including Data Catalog queries. ALWAYS provide params object.

Commands

ListJobs

List Glue jobs.

{ "command": "ListJobs", "params": { "MaxResults": 50 } }

Parameters:

GetJob

Get details of a Glue job.

{ "command": "GetJob", "params": { "JobName": "my-etl-job" } }

Parameters:

GetJobRun

Get details of a specific job run.

{ "command": "GetJobRun", "params": { "JobName": "my-etl-job", "RunId": "jr_abc123" } }

Parameters:

GetJobRuns

List all runs of a job.

{ "command": "GetJobRuns", "params": { "JobName": "my-etl-job", "MaxResults": 20 } }

Parameters:

GetJobBookmark

Get the bookmark state for a job.

{ "command": "GetJobBookmark", "params": { "JobName": "my-etl-job" } }

Parameters:

StartJobRun

Start a Glue job run.

{ "command": "StartJobRun", "params": { "JobName": "my-etl-job", "Arguments": { "--input-path": "s3://my-bucket/input/" } } }

Parameters:

CreateJob

Create a new Glue job.

{
  "command": "CreateJob",
  "params": {
    "Name": "my-new-job",
    "Role": "arn:aws:iam::123456789012:role/GlueRole",
    "Command": { "Name": "glueetl", "ScriptLocation": "s3://my-bucket/scripts/etl.py" },
    "WorkerType": "G.1X",
    "NumberOfWorkers": 10
  }
}

Parameters:

ListTriggers

List Glue triggers.

{ "command": "ListTriggers", "params": { "MaxResults": 50 } }

Parameters:

GetTrigger

Get details of a trigger.

{ "command": "GetTrigger", "params": { "Name": "my-trigger" } }

Parameters:

GetTriggers

List triggers with optional filter.

{ "command": "GetTriggers", "params": { "DependencyJobName": "my-job" } }

Parameters:

ListCrawlers

List Glue crawlers.

{ "command": "ListCrawlers", "params": {} }

Parameters:

GetCrawler

Get details of a crawler.

{ "command": "GetCrawler", "params": { "CrawlerName": "my-crawler" } }

Parameters:

GetCrawlers

List crawlers with details.

{ "command": "GetCrawlers", "params": {} }

Parameters:

ListCrawls

List crawl runs for a crawler.

{ "command": "ListCrawls", "params": { "CrawlerName": "my-crawler" } }

Parameters:

GetDatabase

Get a Glue Data Catalog database.

{ "command": "GetDatabase", "params": { "DatabaseName": "my-database" } }

Parameters:

GetDatabases

List all databases in the Data Catalog.

{ "command": "GetDatabases", "params": {} }

Parameters:

GetTable

Get a table definition from the Data Catalog.

{ "command": "GetTable", "params": { "DatabaseName": "my-database", "Name": "my-table" } }

Parameters:

GetTables

List tables in a database.

{ "command": "GetTables", "params": { "DatabaseName": "my-database" } }

Parameters:

GetPartitions

List partitions for a table.

{ "command": "GetPartitions", "params": { "DatabaseName": "my-database", "TableName": "my-table" } }

Parameters:

GetConnections

List or get Glue connections.

{ "command": "GetConnections", "params": { "ConnectionName": "my-connection" } }

Parameters:

GetTags

Get tags for a Glue resource.

{ "command": "GetTags", "params": { "ResourceArn": "arn:aws:glue:..." } }

Parameters:

Related Services

• •Glue → CloudWatch Logs: Glue job output logs go to /aws-glue/jobs/output and error logs to /aws-glue/jobs/error. Crawler logs go to /aws-glue/crawlers. Use CloudWatchLogTool to read them
• •Glue → S3: Glue jobs read from and write to S3. Crawler targets are often S3 paths. Job scripts are stored in S3. Use S3Tool to inspect
• •Glue → IAM: Jobs require IAM roles. Use IAMTool to inspect the execution role
• •Glue → Data Catalog → Athena/EMR/Redshift: The Glue Data Catalog is shared with Athena, EMR, and Redshift Spectrum
• •Glue → CloudFormation: Glue resources managed by CloudFormation stacks
• •Glue → DynamoDB: Glue can read from DynamoDB tables as data sources