Data engineers and analysts often have to automate their workflows and data processing questions for current data pipes and messages. Amazon Sagemaker Unified Studio is the only AI data and development environment where you can find and access all data in your organization and act on it with the best tools in any use. Amazon Sagemaker Unified Studio provides powerful tools for visual extract, transformation and loading (ETL) and books books. To date, working procedures have required further settings and infrastructure.
Today we are excited that we can present a new function of united planning that simplifies this process. Sagemaker Unified Studio allows you to create ETL flows using a visual interface and write SQL Analytics query for user questions. This new united planning function allows you to plan your visual ETL flows and interviewing books directly from the Sagemaker Unified Studio in the same interface, eliminating the need to visit other consoles or complex configurations. Using Amazon EventBridge Sperller, this feature provides a smooth and easy -to -use planning experience.
In this post, we go through how to plan your visual flows of ETLs and books of queries by just a few clicks, to explore the basic architecture, and demonstrate how this feature can make your data work process more efficient.
Oveview function
Sagemaker Unified Studio Unified Planning is based on top education Amazon Sagemaker. When you configure a new Sagemaker Unified Studio schedule, the New EventBridge plan is an automatic plan created in your AWS account. Eventbridge schedule is configured with API Sagemaker CreateteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteteTraingjob. Sagemaker training works visual flows of ETL or questioning book.
The following diagram illustrates how it works.
Prerequisites
You want to start the instructions, you must have the following assumptions:
- AC ACCOUNT
- On Sagemaker Unified Studio Domain
- On the Sagemaker Unified Studio SA project All the abilities profile. This profile includes Tools Blueprint in which planning is enabled by default. If planning is deactivated, you may need to update the project profile.
- The role of the Sagemaker Unified Studio without borders permission or explicitly allowing GetMedilegroup. New projects have this policy by default. If planning is deactivated, you may need to update the role of your project.
- ROLE PROJECT STUDIO FOR SAGEMaker without project borders or with explicit permission
GetScheduleGroup.
Visual Flow of ETL
Fill in the following steps and plan configuration on the visual flow of ETL:
- On the top menu on the upper offer on the Sagemaker Unified Studio Console Create.
- Under Analysis and integration of datachoose Visual Flows of ETL.
- For Select or create a project to continueChoose your project and choose Continuously.
- Choose your visual flow ETL. If you do not have any visual flows of ETL, Fer to the author Visual ETL flows to Amazon Sagemaker Unified Studio to create a new visual flow of ETL.
- Select Scored icon.
- For NameEnter one name (for example,,
everyday). - For A typical diagramselect Recurring.
- For Valueenter
1. - For Unitchoose days.
- For TimerChoose a time zone.
- Choose Make a plan.
You have successfully configured the plan. Yoke The date and time of the start It is not mentioned, the visual flow of ETL is triggered extremely and then starts once a day.
Edit the plan
You can see the configured plans with the following steps:
- To Stagemaker Unified Studio Console Scroll to Visual Flows of ETL For your project.
- Select Scheme Tab.
- Choose Schedule under Action.
- Edit using your preferences and then select Impose.
Gold Break The Schedule
If you want to pause the schedule, complete the following steps:
- Choose Interruption Scored under Action.
Saturday Scored card, Position The schedule will be updated to Paused.
- You want to summarize the schedule, choose Enable the plan.
Delete
You want to delete a schedule, complete the following steps:
- Choose Delete Scored under Action.
- Choose Delete In the dialogue.
Saturday Scored The card, you can verify that the deleted scheudle disappears.
Plan the flow of the book of Questions
Fill in the following steps and plan configuration on the Question book:
- On the top menu on the upper offer on the Sagemaker Unified Studio Console Create.
- Under Analysis and integration of datachoose Question Editor.
- On the data explorer, under Lakehousechoose
AwsDataCatalog. - Go to the table
venue_event_agg. This table is created in the previous section. - Select the options (three dots) Query with Athena.
- On Action Menu, select Project.
- Choose Save changes.
- On Action Menu, select Make a plan.
- For A typical diagramchoose Recurring.
- For ValueEnter 1.
- For Unitchoose days.
- For TimerChoose a time zone.
- Choose Make a plan.
You have successfully configured the plan. Yoke The date and time of the start It has not been set up, the book of the questions is running ammodery and then launched once a day. If you want to limit your schedule to start the date of the date.
You want to see the configured plans, in the navigation pane they choose Scheduled queries.
You can view a list of planned queries and edit, pause, resume or delete as shown in the previous section.
Clean up
You want to have future fees, clean the resources you created during this passage:
- On Scored Card Visual Flows of ETLselect
everydayPlan and select Delete under Action. The related EventBridge plan is also automatically deleted. - On Sagemaker has a console, select Jobs under Trainingand delete all training on the sagemaker that start
everyday-. - (Optional) to remove the visual flow of ETL to Flows Card Visual Flows of ETLChoose your visual flow ETL and choose Delete under Action.
Conclusion
The new unified planning experience in SageMaker Unified Studio simplifies the automation of the workflow. With unified planning, you can smoothly organize visual flows of ETLs and books of questions in one centralized rental.
Whether you are operating data transformations, weekly analytical questions, or monthly working procedures for reporting, the Unified Planning experience provides a direct path to automation. This capacity allows data teams to focus more on deriving knowledge from their data and less to manage infrastructure and planning configurations.
We recommend that you try this new experience and share your feedback with us. For more information about Sagemaker Unified Studio and its abilities, see documentation or explore other blog posts on visual streams and questioning books.
About the authors
Noritaka Sekiyama It is the main architect of large data for AWS Analytics Services with a strong focus on data engineering. He is responsible for building software artifacts that help customers. In his free time he enjoys cycling on a road bike.
Daniel Obi Is the frontnd engineer in Amazon Sagemaker Unified Studio Team. It is devoted to building intuitive and efficient solutions that increase user experience and technical functionality. Outside his professional work he likes to watch and play basketball.
Vasudevan Vencataramanan is a senior software engineer in Amazon Sagemaker Unified Studio Team. He is responsible for the technical direction of planning and orchestration within the Sagemaker Unified Studio. Outside his professional work he likes to spend time with his child and plays pickball and cricket.
Yuhang huang He is a manager of software development in Amazon Sagemaker Unified Studio Team. It leads the engineering team to design, construction and operation of planning and orchestration skills in the Sagemaker Unifed studio. In her free time he likes to play tennis.
Gal Heyne is the head of technical product manager for AWS Analytics Services with a strong focus on AI/ml and data engineering. It is enthusiastic about developing a deep understanding of customer business and cooperation with engineers in designing easily usable data products.