(Tippapatt/Shutterstock)
AI can drive a bus in terms of investment. But as society fights with their AI introduction, they realize that data problems are what holds them back. This is what Databricks leads to invest in basic data engineering and operating capacities that this week showed this week by launching their designer Lakeflow and Lakebase products this week at their data + AI summit.
Lakeflow, which launched Databricks at its 2024 conference a year, is essentially an ETL tool that allows customers to receive data from different systems, includes databases, cloud sources and business applications, and then automatically deploy and monitor data pipes.
While Lakeflow is great for data engineers and other technical people who know how to coded, it is not necessary for business people to use comfortable. Databricks have heard from their customers that they wanted more advanced tools that allowed them to create data pipes more automated in a more automated way, said Joel Minnick, Marketing Databricks vice president.
“Customers ask us quite a lot” Why is this choice between simplicity and business focus or production? Why does it have to differ? “He said.” And we said, when we looked a little, different things may not. “
The new designer LAKEFLOW Databricks includes GUI and NLP interfaces for data pipe development
Lakeflow Designer is a code -free tool that allows users to create data pipes in two different ways. First, they can use the graphical interface to drag resources and destinations for data pipes using a direct acyclic chart (DAG). Alternatively, they can use a natural language to tell the product the type of data pipe they want to build. In both cases, Lakeflow Design uses Databricks Assistant assistant, a LLM braided by LLM to generate SQL to create real data pipes.
The data pipes built by Lakeflow Designer are treated as well as data pipes built in a traditional way. They both benefit from the same level of security, management and tracking the lines that the code should be generated by man. This is due to integration with the Unity catalog in Lakeflow Designer, MINNICK said.
“Behind the scenes we are talking about two different worlds,” he said. “What is happening when you go through this process, either by dragging and dropping your F, or just asking for what you need is everything that is supported by Lakeflow itself. So because everything that this process is generated for you, all these connections in the Unity catalog make sure it has a sound.
The pipes created with the Lakeflow designer are expandable, so whenever the data engineer can open and work with the pipe in the code-ST interface. Finally, each pipe originally developed by a data engineer working at a lower SQL level can be adjusted to the visual and NLP interfaces.
“At any time, in real time, when you make changes to the EITH side, these changes in the code reflect in the designer and the changes in the designer are renewed in the code,” Minnick said. “So this abyss, which was between the two teams, is now able to disappear.”
Lakeflow Designer will soon enter a private preview. Lakeflow itself is now generally available. The company also announced new connectors for Google Analytics, Servicenow, SQL Server, SharePoint, PostgreSQL and SFTP.
In addition to strengthening data integration and ETL – Long Bane of Cio – Databricks, trying to move the ball forward in another traditional IT discipline: online transaction (OLTP).
Databricks was focused on advanced analysis and AINE was founded in 2013 by the creator of Apache Spark Matei Zaharia and others from the University of California Amplab. But with the launch of Lakebase, it is now getting to Postgres with its registered office at OLTP.
Lakebase is based on an open source, the Postgres Serverless database developed by Neon, which Databricks won last month. As the company explained, the rise of AI agents required a reliable operating database for data location and service.
“Every application, agent, agent, recommendations and automated workflow needs quickly and reliable data in the speed and range of AI agents,” the company said. “This also requires that the operating and analytical system gets closer to reduce the latency between the AI system and to provide companies with current information to make real decisions.”
Databricks said agents will create 90% of databases in the future. The database is revolved on the basis of databricks AI agents on request will be Lakebase, which the company says will be able to run in less than a second.
It is all about bridging the worlds of AI, analysts and operations, said Ali Ghodsi, co -founder and CEO of Databricks.
“We have spent the last few years helping businesses to create AI applications and agents that can consider their propagation data with the Databricks Platform Intelligence Platform,” Statteri. “Now with Lakebase we are creating a new category on the database market: modern Postgres database, deeply integrated with Lakehouse and today’s development chimneys.
Lakebase is now in public preview. You can read more about it on the Databricks blog.
Related items:
Databricks wants to remove the pain of the building and deploy AI agents with bricks
Databricks NABS Neon to solve the narrow section of the AI database
Databricks detect Lakeflow: A uniform and intelligent data engineering tool
(Tagstotranslate) ETL