Apache Spark Now Understands English – iProgrammer

An SDK for Apache Spark has been just released that takes English instructions and compiles them into PySpark objects , rendering Spark more user-friendly and accessible.

It was expected that at some point someone would use Generative AI to compile English to SQL or code. SQL although easy to grasp has been the barrier to Management’s direct interaction with the data silos of the organization.

But in actuality, the developer was the bridge between those two edges. He would get the request of the Management in English or another natural language and translate that to SQL/code in order to generate the necessary report. That role is now played by AI so that even those who do not understand SQL operations can easily use the ability to quickly query business data and generate reports.

But… where the developer had the edge, is that he had knowledge of the schema, the internal details of the database, the business logic which he utilized to construct the query. An AI system without…