dbcli/sqlglot
1
0
Fork 0
Code Issues Activity

Merging upstream version 26.9.0.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-03-09 08:41:51 +01:00
parent cfc68ba563
commit 5699f7334e
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
43 changed files with 27039 additions and 26675 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

36
CHANGELOG.md
View file

@ -1,6 +1,41 @@
Changelog Changelog
========= =========
## [v26.8.0] - 2025-03-03
### :boom: BREAKING CHANGES
- due to [`596b66f`](https://github.com/tobymao/sqlglot/commit/596b66fc289140109db8f689c6e84264d643a47a) - add support for and/2 and or/2 functions *(PR [#4806](https://github.com/tobymao/sqlglot/pull/4806) by [@georgesittas](https://github.com/georgesittas))*:
add support for and/2 and or/2 functions (#4806)
- due to [`eae860c`](https://github.com/tobymao/sqlglot/commit/eae860ce5b59b9e0b791fe79686899efb83df1dd) - expand DISTINCT ON expressions like we do for GROUP/ORDER by *(PR [#4807](https://github.com/tobymao/sqlglot/pull/4807) by [@georgesittas](https://github.com/georgesittas))*:
expand DISTINCT ON expressions like we do for GROUP/ORDER by (#4807)
- due to [`83e6a87`](https://github.com/tobymao/sqlglot/commit/83e6a87f8d233eac6d3bcd3a49451a14dc10e06e) - Parse SHA256 *(PR [#4816](https://github.com/tobymao/sqlglot/pull/4816) by [@VaggelisD](https://github.com/VaggelisD))*:
Parse SHA256 (#4816)
### :sparkles: New Features
- [`50539ce`](https://github.com/tobymao/sqlglot/commit/50539ced46de3949f6a70acdab86129fb50c9385) - **trino**: add support for ON ... ERROR/NULL syntax for JSON_QUERY *(PR [#4805](https://github.com/tobymao/sqlglot/pull/4805) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#3905](https://github.com/TobikoData/sqlmesh/issues/3905) opened by [@darkcofy](https://github.com/darkcofy)*
- [`596b66f`](https://github.com/tobymao/sqlglot/commit/596b66fc289140109db8f689c6e84264d643a47a) - **clickhouse**: add support for and/2 and or/2 functions *(PR [#4806](https://github.com/tobymao/sqlglot/pull/4806) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#4803](https://github.com/tobymao/sqlglot/issues/4803) opened by [@xtess16](https://github.com/xtess16)*
- [`c5bf122`](https://github.com/tobymao/sqlglot/commit/c5bf122a6aa7ca315ad726e6ea3d4a98eebd68d0) - **mysql**: support setting visibility on `ALTER COLUMN`. *(PR [#4809](https://github.com/tobymao/sqlglot/pull/4809) by [@burnison](https://github.com/burnison))*
### :bug: Bug Fixes
- [`6441d00`](https://github.com/tobymao/sqlglot/commit/6441d0041ccec7f1c28763f5775b6195d2049dc6) - orphan node(s) in eliminate_join_marks *(PR [#4808](https://github.com/tobymao/sqlglot/pull/4808) by [@snovik75](https://github.com/snovik75))*
- [`eae860c`](https://github.com/tobymao/sqlglot/commit/eae860ce5b59b9e0b791fe79686899efb83df1dd) - **optimizer**: expand DISTINCT ON expressions like we do for GROUP/ORDER by *(PR [#4807](https://github.com/tobymao/sqlglot/pull/4807) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#4791](https://github.com/tobymao/sqlglot/issues/4791) opened by [@Fosly](https://github.com/Fosly)*
- [`5ef35f2`](https://github.com/tobymao/sqlglot/commit/5ef35f2dc622d96b013a2651c71e1a32933f51cb) - **clickhouse**: unparseable `AggregateFunction(count)` *(PR [#4812](https://github.com/tobymao/sqlglot/pull/4812) by [@pkit](https://github.com/pkit))*
- [`83e6a87`](https://github.com/tobymao/sqlglot/commit/83e6a87f8d233eac6d3bcd3a49451a14dc10e06e) - **duckdb**: Parse SHA256 *(PR [#4816](https://github.com/tobymao/sqlglot/pull/4816) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4815](https://github.com/tobymao/sqlglot/issues/4815) opened by [@muuuuwa](https://github.com/muuuuwa)*
- [`faf6d41`](https://github.com/tobymao/sqlglot/commit/faf6d416afe30bf0bc24649fcceccf79fbfb8ca1) - allow duplicate nodes in matchings *(PR [#4817](https://github.com/tobymao/sqlglot/pull/4817) by [@tobymao](https://github.com/tobymao))*
### :wrench: Chores
- [`e34f3bc`](https://github.com/tobymao/sqlglot/commit/e34f3bc99f832ce2affb3a0297329f3d1cd7244e) - **optimizer**: refactor DISTINCT ON qualification to better match ORDER BY *(PR [#4811](https://github.com/tobymao/sqlglot/pull/4811) by [@georgesittas](https://github.com/georgesittas))*
## [v26.7.0] - 2025-02-26 ## [v26.7.0] - 2025-02-26
### :boom: BREAKING CHANGES ### :boom: BREAKING CHANGES
- due to [`466c839`](https://github.com/tobymao/sqlglot/commit/466c839c2cfc94b398dd619b738df165f2876cdb) - Remove extra MAP bracket and ARRAY wrap *(PR [#4712](https://github.com/tobymao/sqlglot/pull/4712) by [@VaggelisD](https://github.com/VaggelisD))*: - due to [`466c839`](https://github.com/tobymao/sqlglot/commit/466c839c2cfc94b398dd619b738df165f2876cdb) - Remove extra MAP bracket and ARRAY wrap *(PR [#4712](https://github.com/tobymao/sqlglot/pull/4712) by [@VaggelisD](https://github.com/VaggelisD))*:
@ -6003,3 +6038,4 @@ Changelog
[v26.5.0]: https://github.com/tobymao/sqlglot/compare/v26.4.1...v26.5.0 [v26.5.0]: https://github.com/tobymao/sqlglot/compare/v26.4.1...v26.5.0
[v26.6.0]: https://github.com/tobymao/sqlglot/compare/v26.5.0...v26.6.0 [v26.6.0]: https://github.com/tobymao/sqlglot/compare/v26.5.0...v26.6.0
[v26.7.0]: https://github.com/tobymao/sqlglot/compare/v26.6.0...v26.7.0 [v26.7.0]: https://github.com/tobymao/sqlglot/compare/v26.6.0...v26.7.0
[v26.8.0]: https://github.com/tobymao/sqlglot/compare/v26.7.0...v26.8.0

2
docs/search.js
View file

File diff suppressed because one or more lines are too long

12
docs/sqlglot/_version.html
View file

@ -84,8 +84,8 @@
</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span> </span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span> </span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a> </span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a>
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;26.7.0&#39;</span> </span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;26.8.0&#39;</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">26</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> </span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">26</span><span class="p">,</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
</span></pre></div> </span></pre></div>
@ -93,7 +93,7 @@
<section id="__version__"> <section id="__version__">
<div class="attr variable"> <div class="attr variable">
<span class="name">__version__</span><span class="annotation">: str</span> = <span class="name">__version__</span><span class="annotation">: str</span> =
<span class="default_value">&#39;26.7.0&#39;</span> <span class="default_value">&#39;26.8.0&#39;</span>
</div> </div>
@ -105,7 +105,7 @@
<section id="__version_tuple__"> <section id="__version_tuple__">
<div class="attr variable"> <div class="attr variable">
<span class="name">__version_tuple__</span><span class="annotation">: object</span> = <span class="name">__version_tuple__</span><span class="annotation">: object</span> =
<span class="default_value">(26, 7, 0)</span> <span class="default_value">(26, 8, 0)</span>
</div> </div>
@ -117,7 +117,7 @@
<section id="version"> <section id="version">
<div class="attr variable"> <div class="attr variable">
<span class="name">version</span><span class="annotation">: str</span> = <span class="name">version</span><span class="annotation">: str</span> =
<span class="default_value">&#39;26.7.0&#39;</span> <span class="default_value">&#39;26.8.0&#39;</span>
</div> </div>
@ -129,7 +129,7 @@
<section id="version_tuple"> <section id="version_tuple">
<div class="attr variable"> <div class="attr variable">
<span class="name">version_tuple</span><span class="annotation">: object</span> = <span class="name">version_tuple</span><span class="annotation">: object</span> =
<span class="default_value">(26, 7, 0)</span> <span class="default_value">(26, 8, 0)</span>
</div> </div>

58
docs/sqlglot/dialects.html
View file

@ -325,7 +325,7 @@ dialect implementations in order to understand how their various components can
<section id="Athena"> <section id="Athena">
<div class="attr variable"> <div class="attr variable">
<span class="name">Athena</span> = <span class="name">Athena</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462755349024&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234444173472&#39;&gt;</span>
</div> </div>
@ -337,7 +337,7 @@ dialect implementations in order to understand how their various components can
<section id="BigQuery"> <section id="BigQuery">
<div class="attr variable"> <div class="attr variable">
<span class="name">BigQuery</span> = <span class="name">BigQuery</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462741502272&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234434573840&#39;&gt;</span>
</div> </div>
@ -349,7 +349,7 @@ dialect implementations in order to understand how their various components can
<section id="ClickHouse"> <section id="ClickHouse">
<div class="attr variable"> <div class="attr variable">
<span class="name">ClickHouse</span> = <span class="name">ClickHouse</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462754762032&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234434570672&#39;&gt;</span>
</div> </div>
@ -361,7 +361,7 @@ dialect implementations in order to understand how their various components can
<section id="Databricks"> <section id="Databricks">
<div class="attr variable"> <div class="attr variable">
<span class="name">Databricks</span> = <span class="name">Databricks</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462742589632&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234428888496&#39;&gt;</span>
</div> </div>
@ -373,7 +373,7 @@ dialect implementations in order to understand how their various components can
<section id="Doris"> <section id="Doris">
<div class="attr variable"> <div class="attr variable">
<span class="name">Doris</span> = <span class="name">Doris</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462742590208&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234427959280&#39;&gt;</span>
</div> </div>
@ -385,7 +385,7 @@ dialect implementations in order to understand how their various components can
<section id="Drill"> <section id="Drill">
<div class="attr variable"> <div class="attr variable">
<span class="name">Drill</span> = <span class="name">Drill</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462740790416&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234427969360&#39;&gt;</span>
</div> </div>
@ -397,7 +397,7 @@ dialect implementations in order to understand how their various components can
<section id="Druid"> <section id="Druid">
<div class="attr variable"> <div class="attr variable">
<span class="name">Druid</span> = <span class="name">Druid</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462740798336&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234428366560&#39;&gt;</span>
</div> </div>
@ -409,7 +409,7 @@ dialect implementations in order to understand how their various components can
<section id="DuckDB"> <section id="DuckDB">
<div class="attr variable"> <div class="attr variable">
<span class="name">DuckDB</span> = <span class="name">DuckDB</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462741187664&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234428356288&#39;&gt;</span>
</div> </div>
@ -421,7 +421,7 @@ dialect implementations in order to understand how their various components can
<section id="Dune"> <section id="Dune">
<div class="attr variable"> <div class="attr variable">
<span class="name">Dune</span> = <span class="name">Dune</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462745526160&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234443818320&#39;&gt;</span>
</div> </div>
@ -433,7 +433,7 @@ dialect implementations in order to understand how their various components can
<section id="Hive"> <section id="Hive">
<div class="attr variable"> <div class="attr variable">
<span class="name">Hive</span> = <span class="name">Hive</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462750759136&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234443819328&#39;&gt;</span>
</div> </div>
@ -445,7 +445,7 @@ dialect implementations in order to understand how their various components can
<section id="Materialize"> <section id="Materialize">
<div class="attr variable"> <div class="attr variable">
<span class="name">Materialize</span> = <span class="name">Materialize</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462751357952&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234443765840&#39;&gt;</span>
</div> </div>
@ -457,7 +457,7 @@ dialect implementations in order to understand how their various components can
<section id="MySQL"> <section id="MySQL">
<div class="attr variable"> <div class="attr variable">
<span class="name">MySQL</span> = <span class="name">MySQL</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462751363424&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234443756864&#39;&gt;</span>
</div> </div>
@ -469,7 +469,7 @@ dialect implementations in order to understand how their various components can
<section id="Oracle"> <section id="Oracle">
<div class="attr variable"> <div class="attr variable">
<span class="name">Oracle</span> = <span class="name">Oracle</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462755217712&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234443761904&#39;&gt;</span>
</div> </div>
@ -481,7 +481,7 @@ dialect implementations in order to understand how their various components can
<section id="Postgres"> <section id="Postgres">
<div class="attr variable"> <div class="attr variable">
<span class="name">Postgres</span> = <span class="name">Postgres</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462755233024&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234443680944&#39;&gt;</span>
</div> </div>
@ -493,7 +493,7 @@ dialect implementations in order to understand how their various components can
<section id="Presto"> <section id="Presto">
<div class="attr variable"> <div class="attr variable">
<span class="name">Presto</span> = <span class="name">Presto</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462756443568&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234443675808&#39;&gt;</span>
</div> </div>
@ -505,7 +505,7 @@ dialect implementations in order to understand how their various components can
<section id="PRQL"> <section id="PRQL">
<div class="attr variable"> <div class="attr variable">
<span class="name">PRQL</span> = <span class="name">PRQL</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462755122432&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234442518112&#39;&gt;</span>
</div> </div>
@ -517,7 +517,7 @@ dialect implementations in order to understand how their various components can
<section id="Redshift"> <section id="Redshift">
<div class="attr variable"> <div class="attr variable">
<span class="name">Redshift</span> = <span class="name">Redshift</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462755125120&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234442511824&#39;&gt;</span>
</div> </div>
@ -529,7 +529,7 @@ dialect implementations in order to understand how their various components can
<section id="RisingWave"> <section id="RisingWave">
<div class="attr variable"> <div class="attr variable">
<span class="name">RisingWave</span> = <span class="name">RisingWave</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462756817904&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234434918768&#39;&gt;</span>
</div> </div>
@ -541,7 +541,7 @@ dialect implementations in order to understand how their various components can
<section id="Snowflake"> <section id="Snowflake">
<div class="attr variable"> <div class="attr variable">
<span class="name">Snowflake</span> = <span class="name">Snowflake</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462743195648&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234443372672&#39;&gt;</span>
</div> </div>
@ -553,7 +553,7 @@ dialect implementations in order to understand how their various components can
<section id="Spark"> <section id="Spark">
<div class="attr variable"> <div class="attr variable">
<span class="name">Spark</span> = <span class="name">Spark</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462755039024&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234443373296&#39;&gt;</span>
</div> </div>
@ -565,7 +565,7 @@ dialect implementations in order to understand how their various components can
<section id="Spark2"> <section id="Spark2">
<div class="attr variable"> <div class="attr variable">
<span class="name">Spark2</span> = <span class="name">Spark2</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462746722960&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234430652496&#39;&gt;</span>
</div> </div>
@ -577,7 +577,7 @@ dialect implementations in order to understand how their various components can
<section id="SQLite"> <section id="SQLite">
<div class="attr variable"> <div class="attr variable">
<span class="name">SQLite</span> = <span class="name">SQLite</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462742002304&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234445453776&#39;&gt;</span>
</div> </div>
@ -589,7 +589,7 @@ dialect implementations in order to understand how their various components can
<section id="StarRocks"> <section id="StarRocks">
<div class="attr variable"> <div class="attr variable">
<span class="name">StarRocks</span> = <span class="name">StarRocks</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462742011664&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234443357104&#39;&gt;</span>
</div> </div>
@ -601,7 +601,7 @@ dialect implementations in order to understand how their various components can
<section id="Tableau"> <section id="Tableau">
<div class="attr variable"> <div class="attr variable">
<span class="name">Tableau</span> = <span class="name">Tableau</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462741692736&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234443351056&#39;&gt;</span>
</div> </div>
@ -613,7 +613,7 @@ dialect implementations in order to understand how their various components can
<section id="Teradata"> <section id="Teradata">
<div class="attr variable"> <div class="attr variable">
<span class="name">Teradata</span> = <span class="name">Teradata</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462745788496&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234428938992&#39;&gt;</span>
</div> </div>
@ -625,7 +625,7 @@ dialect implementations in order to understand how their various components can
<section id="Trino"> <section id="Trino">
<div class="attr variable"> <div class="attr variable">
<span class="name">Trino</span> = <span class="name">Trino</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462753877488&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234443896096&#39;&gt;</span>
</div> </div>
@ -637,7 +637,7 @@ dialect implementations in order to understand how their various components can
<section id="TSQL"> <section id="TSQL">
<div class="attr variable"> <div class="attr variable">
<span class="name">TSQL</span> = <span class="name">TSQL</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462753885648&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234443893456&#39;&gt;</span>
</div> </div>
@ -649,7 +649,7 @@ dialect implementations in order to understand how their various components can
<section id="Dialect"> <section id="Dialect">
<div class="attr variable"> <div class="attr variable">
<span class="name">Dialect</span> = <span class="name">Dialect</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462739461584&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234426619200&#39;&gt;</span>
</div> </div>
@ -661,7 +661,7 @@ dialect implementations in order to understand how their various components can
<section id="Dialects"> <section id="Dialects">
<div class="attr variable"> <div class="attr variable">
<span class="name">Dialects</span> = <span class="name">Dialects</span> =
<span class="default_value">&lt;MagicMock id=&#39;140462739469504&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140234426627120&#39;&gt;</span>
</div> </div>

1373
docs/sqlglot/diff.html
View file

File diff suppressed because it is too large Load diff

26846
docs/sqlglot/expressions.html
View file

File diff suppressed because one or more lines are too long

8476
docs/sqlglot/generator.html
View file

File diff suppressed because it is too large Load diff

2
docs/sqlglot/helper.html
View file

@ -1920,7 +1920,7 @@ belong to some totally-ordered set.</p>
<section id="DATE_UNITS"> <section id="DATE_UNITS">
<div class="attr variable"> <div class="attr variable">
<span class="name">DATE_UNITS</span> = <span class="name">DATE_UNITS</span> =
<span class="default_value">{&#39;day&#39;, &#39;year_month&#39;, &#39;week&#39;, &#39;quarter&#39;, &#39;year&#39;, &#39;month&#39;}</span> <span class="default_value">{&#39;day&#39;, &#39;month&#39;, &#39;week&#39;, &#39;quarter&#39;, &#39;year_month&#39;, &#39;year&#39;}</span>
</div> </div>

4
docs/sqlglot/optimizer/annotate_types.html
View file

File diff suppressed because one or more lines are too long

2
docs/sqlglot/optimizer/merge_subqueries.html
View file

@ -581,7 +581,7 @@ queries if it would result in multiple table selects in a single query:</p>
<div class="attr variable"> <div class="attr variable">
<span class="name">UNMERGABLE_ARGS</span> = <span class="name">UNMERGABLE_ARGS</span> =
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;sample&#39;, &#39;match&#39;, &#39;sort&#39;, &#39;locks&#39;, &#39;format&#39;, &#39;into&#39;, &#39;limit&#39;, &#39;offset&#39;, &#39;operation_modifiers&#39;, &#39;distinct&#39;, &#39;windows&#39;, &#39;having&#39;, &#39;laterals&#39;, &#39;settings&#39;, &#39;kind&#39;, &#39;distribute&#39;, &#39;connect&#39;, &#39;pivots&#39;, &#39;group&#39;, &#39;cluster&#39;, &#39;prewhere&#39;, &#39;options&#39;, &#39;qualify&#39;, &#39;with&#39;}</span> <label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;operation_modifiers&#39;, &#39;qualify&#39;, &#39;distinct&#39;, &#39;sort&#39;, &#39;cluster&#39;, &#39;limit&#39;, &#39;having&#39;, &#39;connect&#39;, &#39;prewhere&#39;, &#39;match&#39;, &#39;sample&#39;, &#39;into&#39;, &#39;offset&#39;, &#39;settings&#39;, &#39;distribute&#39;, &#39;group&#39;, &#39;pivots&#39;, &#39;options&#39;, &#39;format&#39;, &#39;laterals&#39;, &#39;with&#39;, &#39;windows&#39;, &#39;kind&#39;, &#39;locks&#39;}</span>
</div> </div>

2341
docs/sqlglot/optimizer/qualify_columns.html
View file

File diff suppressed because it is too large Load diff

2235
docs/sqlglot/optimizer/scope.html
View file

File diff suppressed because it is too large Load diff

2
docs/sqlglot/optimizer/simplify.html
View file

@ -3238,7 +3238,7 @@ prefix are statically known.</p>
<div class="attr variable"> <div class="attr variable">
<span class="name">DATETRUNC_COMPARISONS</span> = <span class="name">DATETRUNC_COMPARISONS</span> =
<input id="DATETRUNC_COMPARISONS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="DATETRUNC_COMPARISONS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="../expressions.html#In">sqlglot.expressions.In</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;}</span> <label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#In">sqlglot.expressions.In</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;}</span>
</div> </div>

11668
docs/sqlglot/parser.html
View file

File diff suppressed because one or more lines are too long

2
docs/sqlglot/time.html
View file

File diff suppressed because one or more lines are too long

4
docs/sqlglot/tokens.html
View file

@ -9347,7 +9347,7 @@
<div class="attr variable"> <div class="attr variable">
<span class="name">TOKENS_PRECEDING_HINT</span> = <span class="name">TOKENS_PRECEDING_HINT</span> =
<input id="Tokenizer.TOKENS_PRECEDING_HINT-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="Tokenizer.TOKENS_PRECEDING_HINT-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="Tokenizer.TOKENS_PRECEDING_HINT-view-value"></label><span class="default_value">{&lt;<a href="#TokenType.DELETE">TokenType.DELETE</a>: &#39;DELETE&#39;&gt;, &lt;<a href="#TokenType.SELECT">TokenType.SELECT</a>: &#39;SELECT&#39;&gt;, &lt;<a href="#TokenType.INSERT">TokenType.INSERT</a>: &#39;INSERT&#39;&gt;, &lt;<a href="#TokenType.UPDATE">TokenType.UPDATE</a>: &#39;UPDATE&#39;&gt;}</span> <label class="view-value-button pdoc-button" for="Tokenizer.TOKENS_PRECEDING_HINT-view-value"></label><span class="default_value">{&lt;<a href="#TokenType.SELECT">TokenType.SELECT</a>: &#39;SELECT&#39;&gt;, &lt;<a href="#TokenType.UPDATE">TokenType.UPDATE</a>: &#39;UPDATE&#39;&gt;, &lt;<a href="#TokenType.INSERT">TokenType.INSERT</a>: &#39;INSERT&#39;&gt;, &lt;<a href="#TokenType.DELETE">TokenType.DELETE</a>: &#39;DELETE&#39;&gt;}</span>
</div> </div>
@ -9386,7 +9386,7 @@
<div class="attr variable"> <div class="attr variable">
<span class="name">COMMANDS</span> = <span class="name">COMMANDS</span> =
<input id="Tokenizer.COMMANDS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="Tokenizer.COMMANDS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="Tokenizer.COMMANDS-view-value"></label><span class="default_value">{&lt;<a href="#TokenType.COMMAND">TokenType.COMMAND</a>: &#39;COMMAND&#39;&gt;, &lt;<a href="#TokenType.RENAME">TokenType.RENAME</a>: &#39;RENAME&#39;&gt;, &lt;<a href="#TokenType.EXECUTE">TokenType.EXECUTE</a>: &#39;EXECUTE&#39;&gt;, &lt;<a href="#TokenType.SHOW">TokenType.SHOW</a>: &#39;SHOW&#39;&gt;, &lt;<a href="#TokenType.FETCH">TokenType.FETCH</a>: &#39;FETCH&#39;&gt;}</span> <label class="view-value-button pdoc-button" for="Tokenizer.COMMANDS-view-value"></label><span class="default_value">{&lt;<a href="#TokenType.FETCH">TokenType.FETCH</a>: &#39;FETCH&#39;&gt;, &lt;<a href="#TokenType.RENAME">TokenType.RENAME</a>: &#39;RENAME&#39;&gt;, &lt;<a href="#TokenType.SHOW">TokenType.SHOW</a>: &#39;SHOW&#39;&gt;, &lt;<a href="#TokenType.COMMAND">TokenType.COMMAND</a>: &#39;COMMAND&#39;&gt;, &lt;<a href="#TokenType.EXECUTE">TokenType.EXECUTE</a>: &#39;EXECUTE&#39;&gt;}</span>
</div> </div>

236
docs/sqlglot/transforms.html
View file

@ -1040,67 +1040,66 @@
</span><span id="L-907"><a href="#L-907"><span class="linenos">907</span></a> <span class="nb">len</span><span class="p">(</span><span class="n">marked_column_tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span> </span><span id="L-907"><a href="#L-907"><span class="linenos">907</span></a> <span class="nb">len</span><span class="p">(</span><span class="n">marked_column_tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span>
</span><span id="L-908"><a href="#L-908"><span class="linenos">908</span></a> <span class="p">),</span> <span class="s2">&quot;Columns of only a single table can be marked with (+) in a given binary predicate&quot;</span> </span><span id="L-908"><a href="#L-908"><span class="linenos">908</span></a> <span class="p">),</span> <span class="s2">&quot;Columns of only a single table can be marked with (+) in a given binary predicate&quot;</span>
</span><span id="L-909"><a href="#L-909"><span class="linenos">909</span></a> </span><span id="L-909"><a href="#L-909"><span class="linenos">909</span></a>
</span><span id="L-910"><a href="#L-910"><span class="linenos">910</span></a> <span class="n">join_this</span> <span class="o">=</span> <span class="n">old_joins</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">col</span><span class="o">.</span><span class="n">table</span><span class="p">,</span> <span class="n">query_from</span><span class="p">)</span><span class="o">.</span><span class="n">this</span> </span><span id="L-910"><a href="#L-910"><span class="linenos">910</span></a> <span class="c1"># Add predicate if join already copied, or add join if it is new</span>
</span><span id="L-911"><a href="#L-911"><span class="linenos">911</span></a> <span class="n">new_join</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">join_this</span><span class="p">,</span> <span class="n">on</span><span class="o">=</span><span class="n">join_predicate</span><span class="p">,</span> <span class="n">kind</span><span class="o">=</span><span class="s2">&quot;LEFT&quot;</span><span class="p">)</span> </span><span id="L-911"><a href="#L-911"><span class="linenos">911</span></a> <span class="n">join_this</span> <span class="o">=</span> <span class="n">old_joins</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">col</span><span class="o">.</span><span class="n">table</span><span class="p">,</span> <span class="n">query_from</span><span class="p">)</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-912"><a href="#L-912"><span class="linenos">912</span></a> </span><span id="L-912"><a href="#L-912"><span class="linenos">912</span></a> <span class="n">existing_join</span> <span class="o">=</span> <span class="n">new_joins</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">join_this</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">)</span>
</span><span id="L-913"><a href="#L-913"><span class="linenos">913</span></a> <span class="c1"># Upsert new_join into new_joins dictionary</span> </span><span id="L-913"><a href="#L-913"><span class="linenos">913</span></a> <span class="k">if</span> <span class="n">existing_join</span><span class="p">:</span>
</span><span id="L-914"><a href="#L-914"><span class="linenos">914</span></a> <span class="n">new_join_alias_or_name</span> <span class="o">=</span> <span class="n">new_join</span><span class="o">.</span><span class="n">alias_or_name</span> </span><span id="L-914"><a href="#L-914"><span class="linenos">914</span></a> <span class="n">existing_join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;on&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span><span class="p">(</span><span class="n">existing_join</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;on&quot;</span><span class="p">],</span> <span class="n">join_predicate</span><span class="p">))</span>
</span><span id="L-915"><a href="#L-915"><span class="linenos">915</span></a> <span class="n">existing_join</span> <span class="o">=</span> <span class="n">new_joins</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">new_join_alias_or_name</span><span class="p">)</span> </span><span id="L-915"><a href="#L-915"><span class="linenos">915</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-916"><a href="#L-916"><span class="linenos">916</span></a> <span class="k">if</span> <span class="n">existing_join</span><span class="p">:</span> </span><span id="L-916"><a href="#L-916"><span class="linenos">916</span></a> <span class="n">new_joins</span><span class="p">[</span><span class="n">join_this</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">(</span>
</span><span id="L-917"><a href="#L-917"><span class="linenos">917</span></a> <span class="n">existing_join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;on&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span><span class="p">(</span><span class="n">existing_join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;on&quot;</span><span class="p">),</span> <span class="n">new_join</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;on&quot;</span><span class="p">]))</span> </span><span id="L-917"><a href="#L-917"><span class="linenos">917</span></a> <span class="n">this</span><span class="o">=</span><span class="n">join_this</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">on</span><span class="o">=</span><span class="n">join_predicate</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">kind</span><span class="o">=</span><span class="s2">&quot;LEFT&quot;</span>
</span><span id="L-918"><a href="#L-918"><span class="linenos">918</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="L-918"><a href="#L-918"><span class="linenos">918</span></a> <span class="p">)</span>
</span><span id="L-919"><a href="#L-919"><span class="linenos">919</span></a> <span class="n">new_joins</span><span class="p">[</span><span class="n">new_join_alias_or_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">new_join</span> </span><span id="L-919"><a href="#L-919"><span class="linenos">919</span></a>
</span><span id="L-920"><a href="#L-920"><span class="linenos">920</span></a> </span><span id="L-920"><a href="#L-920"><span class="linenos">920</span></a> <span class="c1"># If the parent of the target predicate is a binary node, then it now has only one child</span>
</span><span id="L-921"><a href="#L-921"><span class="linenos">921</span></a> <span class="c1"># If the parent of the target predicate is a binary node, then it now has only one child</span> </span><span id="L-921"><a href="#L-921"><span class="linenos">921</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate_parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">):</span>
</span><span id="L-922"><a href="#L-922"><span class="linenos">922</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate_parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">):</span> </span><span id="L-922"><a href="#L-922"><span class="linenos">922</span></a> <span class="k">if</span> <span class="n">predicate_parent</span><span class="o">.</span><span class="n">left</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-923"><a href="#L-923"><span class="linenos">923</span></a> <span class="k">if</span> <span class="n">predicate_parent</span><span class="o">.</span><span class="n">left</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span> </span><span id="L-923"><a href="#L-923"><span class="linenos">923</span></a> <span class="n">predicate_parent</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">predicate_parent</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="L-924"><a href="#L-924"><span class="linenos">924</span></a> <span class="n">predicate_parent</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">predicate_parent</span><span class="o">.</span><span class="n">right</span><span class="p">)</span> </span><span id="L-924"><a href="#L-924"><span class="linenos">924</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-925"><a href="#L-925"><span class="linenos">925</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="L-925"><a href="#L-925"><span class="linenos">925</span></a> <span class="n">predicate_parent</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">predicate_parent</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
</span><span id="L-926"><a href="#L-926"><span class="linenos">926</span></a> <span class="n">predicate_parent</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">predicate_parent</span><span class="o">.</span><span class="n">left</span><span class="p">)</span> </span><span id="L-926"><a href="#L-926"><span class="linenos">926</span></a>
</span><span id="L-927"><a href="#L-927"><span class="linenos">927</span></a> </span><span id="L-927"><a href="#L-927"><span class="linenos">927</span></a> <span class="k">if</span> <span class="n">query_from</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">in</span> <span class="n">new_joins</span><span class="p">:</span>
</span><span id="L-928"><a href="#L-928"><span class="linenos">928</span></a> <span class="k">if</span> <span class="n">query_from</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">in</span> <span class="n">new_joins</span><span class="p">:</span> </span><span id="L-928"><a href="#L-928"><span class="linenos">928</span></a> <span class="n">only_old_joins</span> <span class="o">=</span> <span class="n">old_joins</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span> <span class="o">-</span> <span class="n">new_joins</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span>
</span><span id="L-929"><a href="#L-929"><span class="linenos">929</span></a> <span class="n">only_old_joins</span> <span class="o">=</span> <span class="n">old_joins</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span> <span class="o">-</span> <span class="n">new_joins</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span> </span><span id="L-929"><a href="#L-929"><span class="linenos">929</span></a> <span class="k">assert</span> <span class="p">(</span>
</span><span id="L-930"><a href="#L-930"><span class="linenos">930</span></a> <span class="k">assert</span> <span class="p">(</span> </span><span id="L-930"><a href="#L-930"><span class="linenos">930</span></a> <span class="nb">len</span><span class="p">(</span><span class="n">only_old_joins</span><span class="p">)</span> <span class="o">&gt;=</span> <span class="mi">1</span>
</span><span id="L-931"><a href="#L-931"><span class="linenos">931</span></a> <span class="nb">len</span><span class="p">(</span><span class="n">only_old_joins</span><span class="p">)</span> <span class="o">&gt;=</span> <span class="mi">1</span> </span><span id="L-931"><a href="#L-931"><span class="linenos">931</span></a> <span class="p">),</span> <span class="s2">&quot;Cannot determine which table to use in the new FROM clause&quot;</span>
</span><span id="L-932"><a href="#L-932"><span class="linenos">932</span></a> <span class="p">),</span> <span class="s2">&quot;Cannot determine which table to use in the new FROM clause&quot;</span> </span><span id="L-932"><a href="#L-932"><span class="linenos">932</span></a>
</span><span id="L-933"><a href="#L-933"><span class="linenos">933</span></a> </span><span id="L-933"><a href="#L-933"><span class="linenos">933</span></a> <span class="n">new_from_name</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">only_old_joins</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
</span><span id="L-934"><a href="#L-934"><span class="linenos">934</span></a> <span class="n">new_from_name</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">only_old_joins</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> </span><span id="L-934"><a href="#L-934"><span class="linenos">934</span></a> <span class="n">query</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;from&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">old_joins</span><span class="p">[</span><span class="n">new_from_name</span><span class="p">]</span><span class="o">.</span><span class="n">this</span><span class="p">))</span>
</span><span id="L-935"><a href="#L-935"><span class="linenos">935</span></a> <span class="n">query</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;from&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">old_joins</span><span class="p">[</span><span class="n">new_from_name</span><span class="p">]</span><span class="o">.</span><span class="n">this</span><span class="p">))</span> </span><span id="L-935"><a href="#L-935"><span class="linenos">935</span></a>
</span><span id="L-936"><a href="#L-936"><span class="linenos">936</span></a> </span><span id="L-936"><a href="#L-936"><span class="linenos">936</span></a> <span class="k">if</span> <span class="n">new_joins</span><span class="p">:</span>
</span><span id="L-937"><a href="#L-937"><span class="linenos">937</span></a> <span class="k">if</span> <span class="n">new_joins</span><span class="p">:</span> </span><span id="L-937"><a href="#L-937"><span class="linenos">937</span></a> <span class="n">query</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="nb">list</span><span class="p">(</span><span class="n">new_joins</span><span class="o">.</span><span class="n">values</span><span class="p">()))</span>
</span><span id="L-938"><a href="#L-938"><span class="linenos">938</span></a> <span class="n">query</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="nb">list</span><span class="p">(</span><span class="n">new_joins</span><span class="o">.</span><span class="n">values</span><span class="p">()))</span> </span><span id="L-938"><a href="#L-938"><span class="linenos">938</span></a>
</span><span id="L-939"><a href="#L-939"><span class="linenos">939</span></a> </span><span id="L-939"><a href="#L-939"><span class="linenos">939</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">:</span>
</span><span id="L-940"><a href="#L-940"><span class="linenos">940</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">:</span> </span><span id="L-940"><a href="#L-940"><span class="linenos">940</span></a> <span class="n">where</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
</span><span id="L-941"><a href="#L-941"><span class="linenos">941</span></a> <span class="n">where</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span> </span><span id="L-941"><a href="#L-941"><span class="linenos">941</span></a>
</span><span id="L-942"><a href="#L-942"><span class="linenos">942</span></a> </span><span id="L-942"><a href="#L-942"><span class="linenos">942</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-943"><a href="#L-943"><span class="linenos">943</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="L-943"><a href="#L-943"><span class="linenos">943</span></a>
</span><span id="L-944"><a href="#L-944"><span class="linenos">944</span></a> </span><span id="L-944"><a href="#L-944"><span class="linenos">944</span></a>
</span><span id="L-945"><a href="#L-945"><span class="linenos">945</span></a> </span><span id="L-945"><a href="#L-945"><span class="linenos">945</span></a><span class="k">def</span><span class="w"> </span><span class="nf">any_to_exists</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-946"><a href="#L-946"><span class="linenos">946</span></a><span class="k">def</span><span class="w"> </span><span class="nf">any_to_exists</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> </span><span id="L-946"><a href="#L-946"><span class="linenos">946</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-947"><a href="#L-947"><span class="linenos">947</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="L-947"><a href="#L-947"><span class="linenos">947</span></a><span class="sd"> Transform ANY operator to Spark&#39;s EXISTS</span>
</span><span id="L-948"><a href="#L-948"><span class="linenos">948</span></a><span class="sd"> Transform ANY operator to Spark&#39;s EXISTS</span> </span><span id="L-948"><a href="#L-948"><span class="linenos">948</span></a>
</span><span id="L-949"><a href="#L-949"><span class="linenos">949</span></a> </span><span id="L-949"><a href="#L-949"><span class="linenos">949</span></a><span class="sd"> For example,</span>
</span><span id="L-950"><a href="#L-950"><span class="linenos">950</span></a><span class="sd"> For example,</span> </span><span id="L-950"><a href="#L-950"><span class="linenos">950</span></a><span class="sd"> - Postgres: SELECT * FROM tbl WHERE 5 &gt; ANY(tbl.col)</span>
</span><span id="L-951"><a href="#L-951"><span class="linenos">951</span></a><span class="sd"> - Postgres: SELECT * FROM tbl WHERE 5 &gt; ANY(tbl.col)</span> </span><span id="L-951"><a href="#L-951"><span class="linenos">951</span></a><span class="sd"> - Spark: SELECT * FROM tbl WHERE EXISTS(tbl.col, x -&gt; x &lt; 5)</span>
</span><span id="L-952"><a href="#L-952"><span class="linenos">952</span></a><span class="sd"> - Spark: SELECT * FROM tbl WHERE EXISTS(tbl.col, x -&gt; x &lt; 5)</span> </span><span id="L-952"><a href="#L-952"><span class="linenos">952</span></a>
</span><span id="L-953"><a href="#L-953"><span class="linenos">953</span></a> </span><span id="L-953"><a href="#L-953"><span class="linenos">953</span></a><span class="sd"> Both ANY and EXISTS accept queries but currently only array expressions are supported for this</span>
</span><span id="L-954"><a href="#L-954"><span class="linenos">954</span></a><span class="sd"> Both ANY and EXISTS accept queries but currently only array expressions are supported for this</span> </span><span id="L-954"><a href="#L-954"><span class="linenos">954</span></a><span class="sd"> transformation</span>
</span><span id="L-955"><a href="#L-955"><span class="linenos">955</span></a><span class="sd"> transformation</span> </span><span id="L-955"><a href="#L-955"><span class="linenos">955</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-956"><a href="#L-956"><span class="linenos">956</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-956"><a href="#L-956"><span class="linenos">956</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="L-957"><a href="#L-957"><span class="linenos">957</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span> </span><span id="L-957"><a href="#L-957"><span class="linenos">957</span></a> <span class="k">for</span> <span class="nb">any</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Any</span><span class="p">):</span>
</span><span id="L-958"><a href="#L-958"><span class="linenos">958</span></a> <span class="k">for</span> <span class="nb">any</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Any</span><span class="p">):</span> </span><span id="L-958"><a href="#L-958"><span class="linenos">958</span></a> <span class="n">this</span> <span class="o">=</span> <span class="nb">any</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-959"><a href="#L-959"><span class="linenos">959</span></a> <span class="n">this</span> <span class="o">=</span> <span class="nb">any</span><span class="o">.</span><span class="n">this</span> </span><span id="L-959"><a href="#L-959"><span class="linenos">959</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span>
</span><span id="L-960"><a href="#L-960"><span class="linenos">960</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span> </span><span id="L-960"><a href="#L-960"><span class="linenos">960</span></a> <span class="k">continue</span>
</span><span id="L-961"><a href="#L-961"><span class="linenos">961</span></a> <span class="k">continue</span> </span><span id="L-961"><a href="#L-961"><span class="linenos">961</span></a>
</span><span id="L-962"><a href="#L-962"><span class="linenos">962</span></a> </span><span id="L-962"><a href="#L-962"><span class="linenos">962</span></a> <span class="n">binop</span> <span class="o">=</span> <span class="nb">any</span><span class="o">.</span><span class="n">parent</span>
</span><span id="L-963"><a href="#L-963"><span class="linenos">963</span></a> <span class="n">binop</span> <span class="o">=</span> <span class="nb">any</span><span class="o">.</span><span class="n">parent</span> </span><span id="L-963"><a href="#L-963"><span class="linenos">963</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">binop</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">):</span>
</span><span id="L-964"><a href="#L-964"><span class="linenos">964</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">binop</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">):</span> </span><span id="L-964"><a href="#L-964"><span class="linenos">964</span></a> <span class="n">lambda_arg</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="s2">&quot;x&quot;</span><span class="p">)</span>
</span><span id="L-965"><a href="#L-965"><span class="linenos">965</span></a> <span class="n">lambda_arg</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="s2">&quot;x&quot;</span><span class="p">)</span> </span><span id="L-965"><a href="#L-965"><span class="linenos">965</span></a> <span class="nb">any</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">lambda_arg</span><span class="p">)</span>
</span><span id="L-966"><a href="#L-966"><span class="linenos">966</span></a> <span class="nb">any</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">lambda_arg</span><span class="p">)</span> </span><span id="L-966"><a href="#L-966"><span class="linenos">966</span></a> <span class="n">lambda_expr</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Lambda</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">binop</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">lambda_arg</span><span class="p">])</span>
</span><span id="L-967"><a href="#L-967"><span class="linenos">967</span></a> <span class="n">lambda_expr</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Lambda</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">binop</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">lambda_arg</span><span class="p">])</span> </span><span id="L-967"><a href="#L-967"><span class="linenos">967</span></a> <span class="n">binop</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Exists</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">this</span><span class="o">.</span><span class="n">unnest</span><span class="p">(),</span> <span class="n">expression</span><span class="o">=</span><span class="n">lambda_expr</span><span class="p">))</span>
</span><span id="L-968"><a href="#L-968"><span class="linenos">968</span></a> <span class="n">binop</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Exists</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">this</span><span class="o">.</span><span class="n">unnest</span><span class="p">(),</span> <span class="n">expression</span><span class="o">=</span><span class="n">lambda_expr</span><span class="p">))</span> </span><span id="L-968"><a href="#L-968"><span class="linenos">968</span></a>
</span><span id="L-969"><a href="#L-969"><span class="linenos">969</span></a> </span><span id="L-969"><a href="#L-969"><span class="linenos">969</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-970"><a href="#L-970"><span class="linenos">970</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>
@ -2488,40 +2487,39 @@ The corresponding columns are removed from the create statement.</p>
</span><span id="eliminate_join_marks-908"><a href="#eliminate_join_marks-908"><span class="linenos">908</span></a> <span class="nb">len</span><span class="p">(</span><span class="n">marked_column_tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span> </span><span id="eliminate_join_marks-908"><a href="#eliminate_join_marks-908"><span class="linenos">908</span></a> <span class="nb">len</span><span class="p">(</span><span class="n">marked_column_tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span>
</span><span id="eliminate_join_marks-909"><a href="#eliminate_join_marks-909"><span class="linenos">909</span></a> <span class="p">),</span> <span class="s2">&quot;Columns of only a single table can be marked with (+) in a given binary predicate&quot;</span> </span><span id="eliminate_join_marks-909"><a href="#eliminate_join_marks-909"><span class="linenos">909</span></a> <span class="p">),</span> <span class="s2">&quot;Columns of only a single table can be marked with (+) in a given binary predicate&quot;</span>
</span><span id="eliminate_join_marks-910"><a href="#eliminate_join_marks-910"><span class="linenos">910</span></a> </span><span id="eliminate_join_marks-910"><a href="#eliminate_join_marks-910"><span class="linenos">910</span></a>
</span><span id="eliminate_join_marks-911"><a href="#eliminate_join_marks-911"><span class="linenos">911</span></a> <span class="n">join_this</span> <span class="o">=</span> <span class="n">old_joins</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">col</span><span class="o">.</span><span class="n">table</span><span class="p">,</span> <span class="n">query_from</span><span class="p">)</span><span class="o">.</span><span class="n">this</span> </span><span id="eliminate_join_marks-911"><a href="#eliminate_join_marks-911"><span class="linenos">911</span></a> <span class="c1"># Add predicate if join already copied, or add join if it is new</span>
</span><span id="eliminate_join_marks-912"><a href="#eliminate_join_marks-912"><span class="linenos">912</span></a> <span class="n">new_join</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">join_this</span><span class="p">,</span> <span class="n">on</span><span class="o">=</span><span class="n">join_predicate</span><span class="p">,</span> <span class="n">kind</span><span class="o">=</span><span class="s2">&quot;LEFT&quot;</span><span class="p">)</span> </span><span id="eliminate_join_marks-912"><a href="#eliminate_join_marks-912"><span class="linenos">912</span></a> <span class="n">join_this</span> <span class="o">=</span> <span class="n">old_joins</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">col</span><span class="o">.</span><span class="n">table</span><span class="p">,</span> <span class="n">query_from</span><span class="p">)</span><span class="o">.</span><span class="n">this</span>
</span><span id="eliminate_join_marks-913"><a href="#eliminate_join_marks-913"><span class="linenos">913</span></a> </span><span id="eliminate_join_marks-913"><a href="#eliminate_join_marks-913"><span class="linenos">913</span></a> <span class="n">existing_join</span> <span class="o">=</span> <span class="n">new_joins</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">join_this</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">)</span>
</span><span id="eliminate_join_marks-914"><a href="#eliminate_join_marks-914"><span class="linenos">914</span></a> <span class="c1"># Upsert new_join into new_joins dictionary</span> </span><span id="eliminate_join_marks-914"><a href="#eliminate_join_marks-914"><span class="linenos">914</span></a> <span class="k">if</span> <span class="n">existing_join</span><span class="p">:</span>
</span><span id="eliminate_join_marks-915"><a href="#eliminate_join_marks-915"><span class="linenos">915</span></a> <span class="n">new_join_alias_or_name</span> <span class="o">=</span> <span class="n">new_join</span><span class="o">.</span><span class="n">alias_or_name</span> </span><span id="eliminate_join_marks-915"><a href="#eliminate_join_marks-915"><span class="linenos">915</span></a> <span class="n">existing_join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;on&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span><span class="p">(</span><span class="n">existing_join</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;on&quot;</span><span class="p">],</span> <span class="n">join_predicate</span><span class="p">))</span>
</span><span id="eliminate_join_marks-916"><a href="#eliminate_join_marks-916"><span class="linenos">916</span></a> <span class="n">existing_join</span> <span class="o">=</span> <span class="n">new_joins</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">new_join_alias_or_name</span><span class="p">)</span> </span><span id="eliminate_join_marks-916"><a href="#eliminate_join_marks-916"><span class="linenos">916</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="eliminate_join_marks-917"><a href="#eliminate_join_marks-917"><span class="linenos">917</span></a> <span class="k">if</span> <span class="n">existing_join</span><span class="p">:</span> </span><span id="eliminate_join_marks-917"><a href="#eliminate_join_marks-917"><span class="linenos">917</span></a> <span class="n">new_joins</span><span class="p">[</span><span class="n">join_this</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">(</span>
</span><span id="eliminate_join_marks-918"><a href="#eliminate_join_marks-918"><span class="linenos">918</span></a> <span class="n">existing_join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;on&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span><span class="p">(</span><span class="n">existing_join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;on&quot;</span><span class="p">),</span> <span class="n">new_join</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;on&quot;</span><span class="p">]))</span> </span><span id="eliminate_join_marks-918"><a href="#eliminate_join_marks-918"><span class="linenos">918</span></a> <span class="n">this</span><span class="o">=</span><span class="n">join_this</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">on</span><span class="o">=</span><span class="n">join_predicate</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">kind</span><span class="o">=</span><span class="s2">&quot;LEFT&quot;</span>
</span><span id="eliminate_join_marks-919"><a href="#eliminate_join_marks-919"><span class="linenos">919</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="eliminate_join_marks-919"><a href="#eliminate_join_marks-919"><span class="linenos">919</span></a> <span class="p">)</span>
</span><span id="eliminate_join_marks-920"><a href="#eliminate_join_marks-920"><span class="linenos">920</span></a> <span class="n">new_joins</span><span class="p">[</span><span class="n">new_join_alias_or_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">new_join</span> </span><span id="eliminate_join_marks-920"><a href="#eliminate_join_marks-920"><span class="linenos">920</span></a>
</span><span id="eliminate_join_marks-921"><a href="#eliminate_join_marks-921"><span class="linenos">921</span></a> </span><span id="eliminate_join_marks-921"><a href="#eliminate_join_marks-921"><span class="linenos">921</span></a> <span class="c1"># If the parent of the target predicate is a binary node, then it now has only one child</span>
</span><span id="eliminate_join_marks-922"><a href="#eliminate_join_marks-922"><span class="linenos">922</span></a> <span class="c1"># If the parent of the target predicate is a binary node, then it now has only one child</span> </span><span id="eliminate_join_marks-922"><a href="#eliminate_join_marks-922"><span class="linenos">922</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate_parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">):</span>
</span><span id="eliminate_join_marks-923"><a href="#eliminate_join_marks-923"><span class="linenos">923</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate_parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">):</span> </span><span id="eliminate_join_marks-923"><a href="#eliminate_join_marks-923"><span class="linenos">923</span></a> <span class="k">if</span> <span class="n">predicate_parent</span><span class="o">.</span><span class="n">left</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="eliminate_join_marks-924"><a href="#eliminate_join_marks-924"><span class="linenos">924</span></a> <span class="k">if</span> <span class="n">predicate_parent</span><span class="o">.</span><span class="n">left</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span> </span><span id="eliminate_join_marks-924"><a href="#eliminate_join_marks-924"><span class="linenos">924</span></a> <span class="n">predicate_parent</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">predicate_parent</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="eliminate_join_marks-925"><a href="#eliminate_join_marks-925"><span class="linenos">925</span></a> <span class="n">predicate_parent</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">predicate_parent</span><span class="o">.</span><span class="n">right</span><span class="p">)</span> </span><span id="eliminate_join_marks-925"><a href="#eliminate_join_marks-925"><span class="linenos">925</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="eliminate_join_marks-926"><a href="#eliminate_join_marks-926"><span class="linenos">926</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="eliminate_join_marks-926"><a href="#eliminate_join_marks-926"><span class="linenos">926</span></a> <span class="n">predicate_parent</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">predicate_parent</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
</span><span id="eliminate_join_marks-927"><a href="#eliminate_join_marks-927"><span class="linenos">927</span></a> <span class="n">predicate_parent</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">predicate_parent</span><span class="o">.</span><span class="n">left</span><span class="p">)</span> </span><span id="eliminate_join_marks-927"><a href="#eliminate_join_marks-927"><span class="linenos">927</span></a>
</span><span id="eliminate_join_marks-928"><a href="#eliminate_join_marks-928"><span class="linenos">928</span></a> </span><span id="eliminate_join_marks-928"><a href="#eliminate_join_marks-928"><span class="linenos">928</span></a> <span class="k">if</span> <span class="n">query_from</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">in</span> <span class="n">new_joins</span><span class="p">:</span>
</span><span id="eliminate_join_marks-929"><a href="#eliminate_join_marks-929"><span class="linenos">929</span></a> <span class="k">if</span> <span class="n">query_from</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">in</span> <span class="n">new_joins</span><span class="p">:</span> </span><span id="eliminate_join_marks-929"><a href="#eliminate_join_marks-929"><span class="linenos">929</span></a> <span class="n">only_old_joins</span> <span class="o">=</span> <span class="n">old_joins</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span> <span class="o">-</span> <span class="n">new_joins</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span>
</span><span id="eliminate_join_marks-930"><a href="#eliminate_join_marks-930"><span class="linenos">930</span></a> <span class="n">only_old_joins</span> <span class="o">=</span> <span class="n">old_joins</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span> <span class="o">-</span> <span class="n">new_joins</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span> </span><span id="eliminate_join_marks-930"><a href="#eliminate_join_marks-930"><span class="linenos">930</span></a> <span class="k">assert</span> <span class="p">(</span>
</span><span id="eliminate_join_marks-931"><a href="#eliminate_join_marks-931"><span class="linenos">931</span></a> <span class="k">assert</span> <span class="p">(</span> </span><span id="eliminate_join_marks-931"><a href="#eliminate_join_marks-931"><span class="linenos">931</span></a> <span class="nb">len</span><span class="p">(</span><span class="n">only_old_joins</span><span class="p">)</span> <span class="o">&gt;=</span> <span class="mi">1</span>
</span><span id="eliminate_join_marks-932"><a href="#eliminate_join_marks-932"><span class="linenos">932</span></a> <span class="nb">len</span><span class="p">(</span><span class="n">only_old_joins</span><span class="p">)</span> <span class="o">&gt;=</span> <span class="mi">1</span> </span><span id="eliminate_join_marks-932"><a href="#eliminate_join_marks-932"><span class="linenos">932</span></a> <span class="p">),</span> <span class="s2">&quot;Cannot determine which table to use in the new FROM clause&quot;</span>
</span><span id="eliminate_join_marks-933"><a href="#eliminate_join_marks-933"><span class="linenos">933</span></a> <span class="p">),</span> <span class="s2">&quot;Cannot determine which table to use in the new FROM clause&quot;</span> </span><span id="eliminate_join_marks-933"><a href="#eliminate_join_marks-933"><span class="linenos">933</span></a>
</span><span id="eliminate_join_marks-934"><a href="#eliminate_join_marks-934"><span class="linenos">934</span></a> </span><span id="eliminate_join_marks-934"><a href="#eliminate_join_marks-934"><span class="linenos">934</span></a> <span class="n">new_from_name</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">only_old_joins</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
</span><span id="eliminate_join_marks-935"><a href="#eliminate_join_marks-935"><span class="linenos">935</span></a> <span class="n">new_from_name</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">only_old_joins</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> </span><span id="eliminate_join_marks-935"><a href="#eliminate_join_marks-935"><span class="linenos">935</span></a> <span class="n">query</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;from&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">old_joins</span><span class="p">[</span><span class="n">new_from_name</span><span class="p">]</span><span class="o">.</span><span class="n">this</span><span class="p">))</span>
</span><span id="eliminate_join_marks-936"><a href="#eliminate_join_marks-936"><span class="linenos">936</span></a> <span class="n">query</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;from&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">old_joins</span><span class="p">[</span><span class="n">new_from_name</span><span class="p">]</span><span class="o">.</span><span class="n">this</span><span class="p">))</span> </span><span id="eliminate_join_marks-936"><a href="#eliminate_join_marks-936"><span class="linenos">936</span></a>
</span><span id="eliminate_join_marks-937"><a href="#eliminate_join_marks-937"><span class="linenos">937</span></a> </span><span id="eliminate_join_marks-937"><a href="#eliminate_join_marks-937"><span class="linenos">937</span></a> <span class="k">if</span> <span class="n">new_joins</span><span class="p">:</span>
</span><span id="eliminate_join_marks-938"><a href="#eliminate_join_marks-938"><span class="linenos">938</span></a> <span class="k">if</span> <span class="n">new_joins</span><span class="p">:</span> </span><span id="eliminate_join_marks-938"><a href="#eliminate_join_marks-938"><span class="linenos">938</span></a> <span class="n">query</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="nb">list</span><span class="p">(</span><span class="n">new_joins</span><span class="o">.</span><span class="n">values</span><span class="p">()))</span>
</span><span id="eliminate_join_marks-939"><a href="#eliminate_join_marks-939"><span class="linenos">939</span></a> <span class="n">query</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="nb">list</span><span class="p">(</span><span class="n">new_joins</span><span class="o">.</span><span class="n">values</span><span class="p">()))</span> </span><span id="eliminate_join_marks-939"><a href="#eliminate_join_marks-939"><span class="linenos">939</span></a>
</span><span id="eliminate_join_marks-940"><a href="#eliminate_join_marks-940"><span class="linenos">940</span></a> </span><span id="eliminate_join_marks-940"><a href="#eliminate_join_marks-940"><span class="linenos">940</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">:</span>
</span><span id="eliminate_join_marks-941"><a href="#eliminate_join_marks-941"><span class="linenos">941</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">:</span> </span><span id="eliminate_join_marks-941"><a href="#eliminate_join_marks-941"><span class="linenos">941</span></a> <span class="n">where</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
</span><span id="eliminate_join_marks-942"><a href="#eliminate_join_marks-942"><span class="linenos">942</span></a> <span class="n">where</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span> </span><span id="eliminate_join_marks-942"><a href="#eliminate_join_marks-942"><span class="linenos">942</span></a>
</span><span id="eliminate_join_marks-943"><a href="#eliminate_join_marks-943"><span class="linenos">943</span></a> </span><span id="eliminate_join_marks-943"><a href="#eliminate_join_marks-943"><span class="linenos">943</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="eliminate_join_marks-944"><a href="#eliminate_join_marks-944"><span class="linenos">944</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>
@ -2558,31 +2556,31 @@ If this does not hold for a query, consider running <code><a href="optimizer/qua
</div> </div>
<a class="headerlink" href="#any_to_exists"></a> <a class="headerlink" href="#any_to_exists"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="any_to_exists-947"><a href="#any_to_exists-947"><span class="linenos">947</span></a><span class="k">def</span><span class="w"> </span><span class="nf">any_to_exists</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> <div class="pdoc-code codehilite"><pre><span></span><span id="any_to_exists-946"><a href="#any_to_exists-946"><span class="linenos">946</span></a><span class="k">def</span><span class="w"> </span><span class="nf">any_to_exists</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="any_to_exists-948"><a href="#any_to_exists-948"><span class="linenos">948</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="any_to_exists-947"><a href="#any_to_exists-947"><span class="linenos">947</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="any_to_exists-949"><a href="#any_to_exists-949"><span class="linenos">949</span></a><span class="sd"> Transform ANY operator to Spark&#39;s EXISTS</span> </span><span id="any_to_exists-948"><a href="#any_to_exists-948"><span class="linenos">948</span></a><span class="sd"> Transform ANY operator to Spark&#39;s EXISTS</span>
</span><span id="any_to_exists-950"><a href="#any_to_exists-950"><span class="linenos">950</span></a> </span><span id="any_to_exists-949"><a href="#any_to_exists-949"><span class="linenos">949</span></a>
</span><span id="any_to_exists-951"><a href="#any_to_exists-951"><span class="linenos">951</span></a><span class="sd"> For example,</span> </span><span id="any_to_exists-950"><a href="#any_to_exists-950"><span class="linenos">950</span></a><span class="sd"> For example,</span>
</span><span id="any_to_exists-952"><a href="#any_to_exists-952"><span class="linenos">952</span></a><span class="sd"> - Postgres: SELECT * FROM tbl WHERE 5 &gt; ANY(tbl.col)</span> </span><span id="any_to_exists-951"><a href="#any_to_exists-951"><span class="linenos">951</span></a><span class="sd"> - Postgres: SELECT * FROM tbl WHERE 5 &gt; ANY(tbl.col)</span>
</span><span id="any_to_exists-953"><a href="#any_to_exists-953"><span class="linenos">953</span></a><span class="sd"> - Spark: SELECT * FROM tbl WHERE EXISTS(tbl.col, x -&gt; x &lt; 5)</span> </span><span id="any_to_exists-952"><a href="#any_to_exists-952"><span class="linenos">952</span></a><span class="sd"> - Spark: SELECT * FROM tbl WHERE EXISTS(tbl.col, x -&gt; x &lt; 5)</span>
</span><span id="any_to_exists-954"><a href="#any_to_exists-954"><span class="linenos">954</span></a> </span><span id="any_to_exists-953"><a href="#any_to_exists-953"><span class="linenos">953</span></a>
</span><span id="any_to_exists-955"><a href="#any_to_exists-955"><span class="linenos">955</span></a><span class="sd"> Both ANY and EXISTS accept queries but currently only array expressions are supported for this</span> </span><span id="any_to_exists-954"><a href="#any_to_exists-954"><span class="linenos">954</span></a><span class="sd"> Both ANY and EXISTS accept queries but currently only array expressions are supported for this</span>
</span><span id="any_to_exists-956"><a href="#any_to_exists-956"><span class="linenos">956</span></a><span class="sd"> transformation</span> </span><span id="any_to_exists-955"><a href="#any_to_exists-955"><span class="linenos">955</span></a><span class="sd"> transformation</span>
</span><span id="any_to_exists-957"><a href="#any_to_exists-957"><span class="linenos">957</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="any_to_exists-956"><a href="#any_to_exists-956"><span class="linenos">956</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="any_to_exists-958"><a href="#any_to_exists-958"><span class="linenos">958</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span> </span><span id="any_to_exists-957"><a href="#any_to_exists-957"><span class="linenos">957</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="any_to_exists-959"><a href="#any_to_exists-959"><span class="linenos">959</span></a> <span class="k">for</span> <span class="nb">any</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Any</span><span class="p">):</span> </span><span id="any_to_exists-958"><a href="#any_to_exists-958"><span class="linenos">958</span></a> <span class="k">for</span> <span class="nb">any</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Any</span><span class="p">):</span>
</span><span id="any_to_exists-960"><a href="#any_to_exists-960"><span class="linenos">960</span></a> <span class="n">this</span> <span class="o">=</span> <span class="nb">any</span><span class="o">.</span><span class="n">this</span> </span><span id="any_to_exists-959"><a href="#any_to_exists-959"><span class="linenos">959</span></a> <span class="n">this</span> <span class="o">=</span> <span class="nb">any</span><span class="o">.</span><span class="n">this</span>
</span><span id="any_to_exists-961"><a href="#any_to_exists-961"><span class="linenos">961</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span> </span><span id="any_to_exists-960"><a href="#any_to_exists-960"><span class="linenos">960</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span>
</span><span id="any_to_exists-962"><a href="#any_to_exists-962"><span class="linenos">962</span></a> <span class="k">continue</span> </span><span id="any_to_exists-961"><a href="#any_to_exists-961"><span class="linenos">961</span></a> <span class="k">continue</span>
</span><span id="any_to_exists-963"><a href="#any_to_exists-963"><span class="linenos">963</span></a> </span><span id="any_to_exists-962"><a href="#any_to_exists-962"><span class="linenos">962</span></a>
</span><span id="any_to_exists-964"><a href="#any_to_exists-964"><span class="linenos">964</span></a> <span class="n">binop</span> <span class="o">=</span> <span class="nb">any</span><span class="o">.</span><span class="n">parent</span> </span><span id="any_to_exists-963"><a href="#any_to_exists-963"><span class="linenos">963</span></a> <span class="n">binop</span> <span class="o">=</span> <span class="nb">any</span><span class="o">.</span><span class="n">parent</span>
</span><span id="any_to_exists-965"><a href="#any_to_exists-965"><span class="linenos">965</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">binop</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">):</span> </span><span id="any_to_exists-964"><a href="#any_to_exists-964"><span class="linenos">964</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">binop</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">):</span>
</span><span id="any_to_exists-966"><a href="#any_to_exists-966"><span class="linenos">966</span></a> <span class="n">lambda_arg</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="s2">&quot;x&quot;</span><span class="p">)</span> </span><span id="any_to_exists-965"><a href="#any_to_exists-965"><span class="linenos">965</span></a> <span class="n">lambda_arg</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="s2">&quot;x&quot;</span><span class="p">)</span>
</span><span id="any_to_exists-967"><a href="#any_to_exists-967"><span class="linenos">967</span></a> <span class="nb">any</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">lambda_arg</span><span class="p">)</span> </span><span id="any_to_exists-966"><a href="#any_to_exists-966"><span class="linenos">966</span></a> <span class="nb">any</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">lambda_arg</span><span class="p">)</span>
</span><span id="any_to_exists-968"><a href="#any_to_exists-968"><span class="linenos">968</span></a> <span class="n">lambda_expr</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Lambda</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">binop</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">lambda_arg</span><span class="p">])</span> </span><span id="any_to_exists-967"><a href="#any_to_exists-967"><span class="linenos">967</span></a> <span class="n">lambda_expr</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Lambda</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">binop</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">expressions</span><span class="o">=</span><span class="p">[</span><span class="n">lambda_arg</span><span class="p">])</span>
</span><span id="any_to_exists-969"><a href="#any_to_exists-969"><span class="linenos">969</span></a> <span class="n">binop</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Exists</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">this</span><span class="o">.</span><span class="n">unnest</span><span class="p">(),</span> <span class="n">expression</span><span class="o">=</span><span class="n">lambda_expr</span><span class="p">))</span> </span><span id="any_to_exists-968"><a href="#any_to_exists-968"><span class="linenos">968</span></a> <span class="n">binop</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Exists</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">this</span><span class="o">.</span><span class="n">unnest</span><span class="p">(),</span> <span class="n">expression</span><span class="o">=</span><span class="n">lambda_expr</span><span class="p">))</span>
</span><span id="any_to_exists-970"><a href="#any_to_exists-970"><span class="linenos">970</span></a> </span><span id="any_to_exists-969"><a href="#any_to_exists-969"><span class="linenos">969</span></a>
</span><span id="any_to_exists-971"><a href="#any_to_exists-971"><span class="linenos">971</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="any_to_exists-970"><a href="#any_to_exists-970"><span class="linenos">970</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>

4
pdoc/cli.py
View file

@ -8,6 +8,10 @@ from pdoc.__main__ import cli, parser
# Need this import or else import_module doesn't work # Need this import or else import_module doesn't work
import sqlglot # noqa import sqlglot # noqa
from sqlglot.dialects import *
# Load all dialects up front because lazy loading breaks pdoc's dynamic importing
sqlglot.dialects.__all__ = [globals()[attr_name] for attr_name in sqlglot.dialects.__all__]
def mocked_import(*args, **kwargs): def mocked_import(*args, **kwargs):

6
sqlglot/dialects/bigquery.py
View file

@ -27,6 +27,7 @@ from sqlglot.dialects.dialect import (
ts_or_ds_add_cast, ts_or_ds_add_cast,
unit_to_var, unit_to_var,
strposition_sql, strposition_sql,
groupconcat_sql,
) )
from sqlglot.helper import seq_get, split_num_words from sqlglot.helper import seq_get, split_num_words
from sqlglot.tokens import TokenType from sqlglot.tokens import TokenType
@ -902,7 +903,9 @@ class BigQuery(Dialect):
"DATETIME", self.func("TIMESTAMP", e.this, e.args.get("zone")), "'UTC'" "DATETIME", self.func("TIMESTAMP", e.this, e.args.get("zone")), "'UTC'"
), ),
exp.GenerateSeries: rename_func("GENERATE_ARRAY"), exp.GenerateSeries: rename_func("GENERATE_ARRAY"),
exp.GroupConcat: rename_func("STRING_AGG"), exp.GroupConcat: lambda self, e: groupconcat_sql(
self, e, func_name="STRING_AGG", within_group=False
),
exp.Hex: lambda self, e: self.func("UPPER", self.func("TO_HEX", self.sql(e, "this"))), exp.Hex: lambda self, e: self.func("UPPER", self.func("TO_HEX", self.sql(e, "this"))),
exp.HexString: lambda self, e: self.hexstring_sql(e, binary_function_repr="FROM_HEX"), exp.HexString: lambda self, e: self.hexstring_sql(e, binary_function_repr="FROM_HEX"),
exp.If: if_sql(false_value="NULL"), exp.If: if_sql(false_value="NULL"),
@ -989,6 +992,7 @@ class BigQuery(Dialect):
exp.DataType.Type.BIGDECIMAL: "BIGNUMERIC", exp.DataType.Type.BIGDECIMAL: "BIGNUMERIC",
exp.DataType.Type.BIGINT: "INT64", exp.DataType.Type.BIGINT: "INT64",
exp.DataType.Type.BINARY: "BYTES", exp.DataType.Type.BINARY: "BYTES",
exp.DataType.Type.BLOB: "BYTES",
exp.DataType.Type.BOOLEAN: "BOOL", exp.DataType.Type.BOOLEAN: "BOOL",
exp.DataType.Type.CHAR: "STRING", exp.DataType.Type.CHAR: "STRING",
exp.DataType.Type.DECIMAL: "NUMERIC", exp.DataType.Type.DECIMAL: "NUMERIC",

1
sqlglot/dialects/clickhouse.py
View file

@ -920,6 +920,7 @@ class ClickHouse(Dialect):
ARRAY_SIZE_NAME = "LENGTH" ARRAY_SIZE_NAME = "LENGTH"
STRING_TYPE_MAPPING = { STRING_TYPE_MAPPING = {
exp.DataType.Type.BLOB: "String",
exp.DataType.Type.CHAR: "String", exp.DataType.Type.CHAR: "String",
exp.DataType.Type.LONGBLOB: "String", exp.DataType.Type.LONGBLOB: "String",
exp.DataType.Type.LONGTEXT: "String", exp.DataType.Type.LONGTEXT: "String",

33
sqlglot/dialects/dialect.py
View file

@ -1651,7 +1651,7 @@ def build_json_extract_path(
return expr_type.from_arg_list(args) return expr_type.from_arg_list(args)
text = arg.name text = arg.name
if is_int(text): if is_int(text) and (not arrow_req_json_type or not arg.is_string):
index = int(text) index = int(text)
segments.append( segments.append(
exp.JSONPathSubscript(this=index if zero_based_indexing else index - 1) exp.JSONPathSubscript(this=index if zero_based_indexing else index - 1)
@ -1825,3 +1825,34 @@ def no_make_interval_sql(self: Generator, expression: exp.MakeInterval, sep: str
def length_or_char_length_sql(self: Generator, expression: exp.Length) -> str: def length_or_char_length_sql(self: Generator, expression: exp.Length) -> str:
length_func = "LENGTH" if expression.args.get("binary") else "CHAR_LENGTH" length_func = "LENGTH" if expression.args.get("binary") else "CHAR_LENGTH"
return self.func(length_func, expression.this) return self.func(length_func, expression.this)
def groupconcat_sql(
self: Generator,
expression: exp.GroupConcat,
func_name="LISTAGG",
sep: str = ",",
within_group: bool = True,
on_overflow: bool = False,
) -> str:
this = expression.this
separator = expression.args.get("separator") or exp.Literal.string(sep)
on_overflow_sql = self.sql(expression, "on_overflow")
on_overflow_sql = f" ON OVERFLOW {on_overflow_sql}" if (on_overflow and on_overflow_sql) else ""
order = this.find(exp.Order)
if order and order.this:
this = order.this.pop()
args = self.format_args(this, f"{separator}{on_overflow_sql}")
listagg: exp.Expression = exp.Anonymous(this=func_name, expressions=[args])
if order:
if within_group:
listagg = exp.WithinGroup(this=listagg, expression=order)
else:
listagg.set("expressions", [f"{args}{self.sql(expression=expression.this)}"])
return self.sql(listagg)

13
sqlglot/dialects/duckdb.py
View file

@ -36,6 +36,7 @@ from sqlglot.dialects.dialect import (
build_regexp_extract, build_regexp_extract,
explode_to_unnest_sql, explode_to_unnest_sql,
no_make_interval_sql, no_make_interval_sql,
groupconcat_sql,
) )
from sqlglot.generator import unsupported_args from sqlglot.generator import unsupported_args
from sqlglot.helper import seq_get from sqlglot.helper import seq_get
@ -427,7 +428,12 @@ class DuckDB(Dialect):
FUNCTIONS.pop("DATE_SUB") FUNCTIONS.pop("DATE_SUB")
FUNCTIONS.pop("GLOB") FUNCTIONS.pop("GLOB")
FUNCTION_PARSERS = parser.Parser.FUNCTION_PARSERS.copy() FUNCTION_PARSERS = {
**parser.Parser.FUNCTION_PARSERS,
**dict.fromkeys(
("GROUP_CONCAT", "LISTAGG", "STRINGAGG"), lambda self: self._parse_string_agg()
),
}
FUNCTION_PARSERS.pop("DECODE") FUNCTION_PARSERS.pop("DECODE")
NO_PAREN_FUNCTION_PARSERS = { NO_PAREN_FUNCTION_PARSERS = {
@ -583,6 +589,7 @@ class DuckDB(Dialect):
exp.Encode: lambda self, e: encode_decode_sql(self, e, "ENCODE", replace=False), exp.Encode: lambda self, e: encode_decode_sql(self, e, "ENCODE", replace=False),
exp.GenerateDateArray: _generate_datetime_array_sql, exp.GenerateDateArray: _generate_datetime_array_sql,
exp.GenerateTimestampArray: _generate_datetime_array_sql, exp.GenerateTimestampArray: _generate_datetime_array_sql,
exp.GroupConcat: lambda self, e: groupconcat_sql(self, e, within_group=False),
exp.HexString: lambda self, e: self.hexstring_sql(e, binary_function_repr="FROM_HEX"), exp.HexString: lambda self, e: self.hexstring_sql(e, binary_function_repr="FROM_HEX"),
exp.Explode: rename_func("UNNEST"), exp.Explode: rename_func("UNNEST"),
exp.IntDiv: lambda self, e: self.binary(e, "//"), exp.IntDiv: lambda self, e: self.binary(e, "//"),
@ -1058,3 +1065,7 @@ class DuckDB(Dialect):
self.unsupported("Only integer formats are supported by NumberToStr") self.unsupported("Only integer formats are supported by NumberToStr")
return self.function_fallback_sql(expression) return self.function_fallback_sql(expression)
def autoincrementcolumnconstraint_sql(self, _) -> str:
self.unsupported("The AUTOINCREMENT column constraint is not supported by DuckDB")
return ""

2
sqlglot/dialects/hive.py
View file

@ -486,6 +486,7 @@ class Hive(Dialect):
TYPE_MAPPING = { TYPE_MAPPING = {
**generator.Generator.TYPE_MAPPING, **generator.Generator.TYPE_MAPPING,
exp.DataType.Type.BIT: "BOOLEAN", exp.DataType.Type.BIT: "BOOLEAN",
exp.DataType.Type.BLOB: "BINARY",
exp.DataType.Type.DATETIME: "TIMESTAMP", exp.DataType.Type.DATETIME: "TIMESTAMP",
exp.DataType.Type.ROWVERSION: "BINARY", exp.DataType.Type.ROWVERSION: "BINARY",
exp.DataType.Type.TEXT: "STRING", exp.DataType.Type.TEXT: "STRING",
@ -517,6 +518,7 @@ class Hive(Dialect):
e: f"TO_DATE(CAST({self.sql(e, 'this')} AS STRING), {Hive.DATEINT_FORMAT})", e: f"TO_DATE(CAST({self.sql(e, 'this')} AS STRING), {Hive.DATEINT_FORMAT})",
exp.FileFormatProperty: lambda self, exp.FileFormatProperty: lambda self,
e: f"STORED AS {self.sql(e, 'this') if isinstance(e.this, exp.InputOutputFormat) else e.name.upper()}", e: f"STORED AS {self.sql(e, 'this') if isinstance(e.this, exp.InputOutputFormat) else e.name.upper()}",
exp.StorageHandlerProperty: lambda self, e: f"STORED BY {self.sql(e, 'this')}",
exp.FromBase64: rename_func("UNBASE64"), exp.FromBase64: rename_func("UNBASE64"),
exp.GenerateSeries: sequence_sql, exp.GenerateSeries: sequence_sql,
exp.GenerateDateArray: sequence_sql, exp.GenerateDateArray: sequence_sql,

2
sqlglot/dialects/mysql.py
View file

@ -192,6 +192,7 @@ class MySQL(Dialect):
"CHARSET": TokenType.CHARACTER_SET, "CHARSET": TokenType.CHARACTER_SET,
# The DESCRIBE and EXPLAIN statements are synonyms. # The DESCRIBE and EXPLAIN statements are synonyms.
# https://dev.mysql.com/doc/refman/8.4/en/explain.html # https://dev.mysql.com/doc/refman/8.4/en/explain.html
"BLOB": TokenType.BLOB,
"EXPLAIN": TokenType.DESCRIBE, "EXPLAIN": TokenType.DESCRIBE,
"FORCE": TokenType.FORCE, "FORCE": TokenType.FORCE,
"IGNORE": TokenType.IGNORE, "IGNORE": TokenType.IGNORE,
@ -830,6 +831,7 @@ class MySQL(Dialect):
TYPE_MAPPING.pop(exp.DataType.Type.MEDIUMTEXT) TYPE_MAPPING.pop(exp.DataType.Type.MEDIUMTEXT)
TYPE_MAPPING.pop(exp.DataType.Type.LONGTEXT) TYPE_MAPPING.pop(exp.DataType.Type.LONGTEXT)
TYPE_MAPPING.pop(exp.DataType.Type.TINYTEXT) TYPE_MAPPING.pop(exp.DataType.Type.TINYTEXT)
TYPE_MAPPING.pop(exp.DataType.Type.BLOB)
TYPE_MAPPING.pop(exp.DataType.Type.MEDIUMBLOB) TYPE_MAPPING.pop(exp.DataType.Type.MEDIUMBLOB)
TYPE_MAPPING.pop(exp.DataType.Type.LONGBLOB) TYPE_MAPPING.pop(exp.DataType.Type.LONGBLOB)
TYPE_MAPPING.pop(exp.DataType.Type.TINYBLOB) TYPE_MAPPING.pop(exp.DataType.Type.TINYBLOB)

2
sqlglot/dialects/oracle.py
View file

@ -153,6 +153,7 @@ class Oracle(Dialect):
and self.expression(exp.TemporaryProperty, this="GLOBAL"), and self.expression(exp.TemporaryProperty, this="GLOBAL"),
"PRIVATE": lambda self: self._match_text_seq("TEMPORARY") "PRIVATE": lambda self: self._match_text_seq("TEMPORARY")
and self.expression(exp.TemporaryProperty, this="PRIVATE"), and self.expression(exp.TemporaryProperty, this="PRIVATE"),
"FORCE": lambda self: self.expression(exp.ForceProperty),
} }
QUERY_MODIFIER_PARSERS = { QUERY_MODIFIER_PARSERS = {
@ -282,6 +283,7 @@ class Oracle(Dialect):
exp.DataType.Type.VARBINARY: "BLOB", exp.DataType.Type.VARBINARY: "BLOB",
exp.DataType.Type.ROWVERSION: "BLOB", exp.DataType.Type.ROWVERSION: "BLOB",
} }
TYPE_MAPPING.pop(exp.DataType.Type.BLOB)
TRANSFORMS = { TRANSFORMS = {
**generator.Generator.TRANSFORMS, **generator.Generator.TRANSFORMS,

19
sqlglot/dialects/postgres.py
View file

@ -34,6 +34,7 @@ from sqlglot.dialects.dialect import (
ts_or_ds_add_cast, ts_or_ds_add_cast,
strposition_sql, strposition_sql,
count_if_to_sum, count_if_to_sum,
groupconcat_sql,
) )
from sqlglot.generator import unsupported_args from sqlglot.generator import unsupported_args
from sqlglot.helper import is_int, seq_get from sqlglot.helper import is_int, seq_get
@ -109,19 +110,6 @@ def _substring_sql(self: Postgres.Generator, expression: exp.Substring) -> str:
return f"SUBSTRING({this}{from_part}{for_part})" return f"SUBSTRING({this}{from_part}{for_part})"
def _string_agg_sql(self: Postgres.Generator, expression: exp.GroupConcat) -> str:
separator = expression.args.get("separator") or exp.Literal.string(",")
order = ""
this = expression.this
if isinstance(this, exp.Order):
if this.this:
this = this.this.pop()
order = self.sql(expression.this) # Order has a leading space
return f"STRING_AGG({self.format_args(this, separator)}{order})"
def _auto_increment_to_serial(expression: exp.Expression) -> exp.Expression: def _auto_increment_to_serial(expression: exp.Expression) -> exp.Expression:
auto = expression.find(exp.AutoIncrementColumnConstraint) auto = expression.find(exp.AutoIncrementColumnConstraint)
@ -539,6 +527,7 @@ class Postgres(Dialect):
exp.DataType.Type.VARBINARY: "BYTEA", exp.DataType.Type.VARBINARY: "BYTEA",
exp.DataType.Type.ROWVERSION: "BYTEA", exp.DataType.Type.ROWVERSION: "BYTEA",
exp.DataType.Type.DATETIME: "TIMESTAMP", exp.DataType.Type.DATETIME: "TIMESTAMP",
exp.DataType.Type.BLOB: "BYTEA",
} }
TRANSFORMS = { TRANSFORMS = {
@ -557,7 +546,9 @@ class Postgres(Dialect):
exp.DateSub: _date_add_sql("-"), exp.DateSub: _date_add_sql("-"),
exp.Explode: rename_func("UNNEST"), exp.Explode: rename_func("UNNEST"),
exp.ExplodingGenerateSeries: rename_func("GENERATE_SERIES"), exp.ExplodingGenerateSeries: rename_func("GENERATE_SERIES"),
exp.GroupConcat: _string_agg_sql, exp.GroupConcat: lambda self, e: groupconcat_sql(
self, e, func_name="STRING_AGG", within_group=False
),
exp.IntDiv: rename_func("DIV"), exp.IntDiv: rename_func("DIV"),
exp.JSONExtract: _json_extract_sql("JSON_EXTRACT_PATH", "->"), exp.JSONExtract: _json_extract_sql("JSON_EXTRACT_PATH", "->"),
exp.JSONExtractScalar: _json_extract_sql("JSON_EXTRACT_PATH_TEXT", "->>"), exp.JSONExtractScalar: _json_extract_sql("JSON_EXTRACT_PATH_TEXT", "->>"),

1
sqlglot/dialects/redshift.py
View file

@ -166,6 +166,7 @@ class Redshift(Postgres):
TYPE_MAPPING = { TYPE_MAPPING = {
**Postgres.Generator.TYPE_MAPPING, **Postgres.Generator.TYPE_MAPPING,
exp.DataType.Type.BINARY: "VARBYTE", exp.DataType.Type.BINARY: "VARBYTE",
exp.DataType.Type.BLOB: "VARBYTE",
exp.DataType.Type.INT: "INTEGER", exp.DataType.Type.INT: "INTEGER",
exp.DataType.Type.TIMETZ: "TIME", exp.DataType.Type.TIMETZ: "TIME",
exp.DataType.Type.TIMESTAMPTZ: "TIMESTAMP", exp.DataType.Type.TIMESTAMPTZ: "TIMESTAMP",

20
sqlglot/dialects/snowflake.py
View file

@ -26,6 +26,7 @@ from sqlglot.dialects.dialect import (
strposition_sql, strposition_sql,
timestampdiff_sql, timestampdiff_sql,
no_make_interval_sql, no_make_interval_sql,
groupconcat_sql,
) )
from sqlglot.generator import unsupported_args from sqlglot.generator import unsupported_args
from sqlglot.helper import flatten, is_float, is_int, seq_get from sqlglot.helper import flatten, is_float, is_int, seq_get
@ -439,7 +440,6 @@ class Snowflake(Dialect):
), ),
"LEN": lambda args: exp.Length(this=seq_get(args, 0), binary=True), "LEN": lambda args: exp.Length(this=seq_get(args, 0), binary=True),
"LENGTH": lambda args: exp.Length(this=seq_get(args, 0), binary=True), "LENGTH": lambda args: exp.Length(this=seq_get(args, 0), binary=True),
"LISTAGG": exp.GroupConcat.from_arg_list,
"NULLIFZERO": _build_if_from_nullifzero, "NULLIFZERO": _build_if_from_nullifzero,
"OBJECT_CONSTRUCT": _build_object_construct, "OBJECT_CONSTRUCT": _build_object_construct,
"REGEXP_EXTRACT_ALL": _build_regexp_extract(exp.RegexpExtractAll), "REGEXP_EXTRACT_ALL": _build_regexp_extract(exp.RegexpExtractAll),
@ -482,6 +482,7 @@ class Snowflake(Dialect):
**parser.Parser.FUNCTION_PARSERS, **parser.Parser.FUNCTION_PARSERS,
"DATE_PART": lambda self: self._parse_date_part(), "DATE_PART": lambda self: self._parse_date_part(),
"OBJECT_CONSTRUCT_KEEP_NULL": lambda self: self._parse_json_object(), "OBJECT_CONSTRUCT_KEEP_NULL": lambda self: self._parse_json_object(),
"LISTAGG": lambda self: self._parse_string_agg(),
} }
FUNCTION_PARSERS.pop("TRIM") FUNCTION_PARSERS.pop("TRIM")
@ -505,6 +506,7 @@ class Snowflake(Dialect):
STATEMENT_PARSERS = { STATEMENT_PARSERS = {
**parser.Parser.STATEMENT_PARSERS, **parser.Parser.STATEMENT_PARSERS,
TokenType.PUT: lambda self: self._parse_put(),
TokenType.SHOW: lambda self: self._parse_show(), TokenType.SHOW: lambda self: self._parse_show(),
} }
@ -805,6 +807,17 @@ class Snowflake(Dialect):
}, },
) )
def _parse_put(self) -> exp.Put | exp.Command:
if self._curr.token_type != TokenType.STRING:
return self._parse_as_command(self._prev)
return self.expression(
exp.Put,
this=self._parse_string(),
target=self._parse_location_path(),
properties=self._parse_properties(),
)
def _parse_location_property(self) -> exp.LocationProperty: def _parse_location_property(self) -> exp.LocationProperty:
self._match(TokenType.EQ) self._match(TokenType.EQ)
return self.expression(exp.LocationProperty, this=self._parse_location_path()) return self.expression(exp.LocationProperty, this=self._parse_location_path())
@ -860,6 +873,7 @@ class Snowflake(Dialect):
KEYWORDS = { KEYWORDS = {
**tokens.Tokenizer.KEYWORDS, **tokens.Tokenizer.KEYWORDS,
"FILE://": TokenType.URI_START,
"BYTEINT": TokenType.INT, "BYTEINT": TokenType.INT,
"CHAR VARYING": TokenType.VARCHAR, "CHAR VARYING": TokenType.VARCHAR,
"CHARACTER VARYING": TokenType.VARCHAR, "CHARACTER VARYING": TokenType.VARCHAR,
@ -870,7 +884,7 @@ class Snowflake(Dialect):
"MATCH_RECOGNIZE": TokenType.MATCH_RECOGNIZE, "MATCH_RECOGNIZE": TokenType.MATCH_RECOGNIZE,
"MINUS": TokenType.EXCEPT, "MINUS": TokenType.EXCEPT,
"NCHAR VARYING": TokenType.VARCHAR, "NCHAR VARYING": TokenType.VARCHAR,
"PUT": TokenType.COMMAND, "PUT": TokenType.PUT,
"REMOVE": TokenType.COMMAND, "REMOVE": TokenType.COMMAND,
"RM": TokenType.COMMAND, "RM": TokenType.COMMAND,
"SAMPLE": TokenType.TABLE_SAMPLE, "SAMPLE": TokenType.TABLE_SAMPLE,
@ -950,7 +964,7 @@ class Snowflake(Dialect):
exp.GenerateSeries: lambda self, e: self.func( exp.GenerateSeries: lambda self, e: self.func(
"ARRAY_GENERATE_RANGE", e.args["start"], e.args["end"] + 1, e.args.get("step") "ARRAY_GENERATE_RANGE", e.args["start"], e.args["end"] + 1, e.args.get("step")
), ),
exp.GroupConcat: rename_func("LISTAGG"), exp.GroupConcat: lambda self, e: groupconcat_sql(self, e, sep=""),
exp.If: if_sql(name="IFF", false_value="NULL"), exp.If: if_sql(name="IFF", false_value="NULL"),
exp.JSONExtractArray: _json_extract_value_array_sql, exp.JSONExtractArray: _json_extract_value_array_sql,
exp.JSONExtractScalar: lambda self, e: self.func( exp.JSONExtractScalar: lambda self, e: self.func(

1
sqlglot/dialects/sqlite.py
View file

@ -165,6 +165,7 @@ class SQLite(Dialect):
exp.DataType.Type.BINARY: "BLOB", exp.DataType.Type.BINARY: "BLOB",
exp.DataType.Type.VARBINARY: "BLOB", exp.DataType.Type.VARBINARY: "BLOB",
} }
TYPE_MAPPING.pop(exp.DataType.Type.BLOB)
TOKEN_MAPPING = { TOKEN_MAPPING = {
TokenType.AUTO_INCREMENT: "AUTOINCREMENT", TokenType.AUTO_INCREMENT: "AUTOINCREMENT",

22
sqlglot/dialects/trino.py
View file

@ -1,7 +1,12 @@
from __future__ import annotations from __future__ import annotations
from sqlglot import exp, parser from sqlglot import exp, parser
from sqlglot.dialects.dialect import merge_without_target_sql, trim_sql, timestrtotime_sql from sqlglot.dialects.dialect import (
merge_without_target_sql,
trim_sql,
timestrtotime_sql,
groupconcat_sql,
)
from sqlglot.dialects.presto import Presto from sqlglot.dialects.presto import Presto
from sqlglot.tokens import TokenType from sqlglot.tokens import TokenType
import typing as t import typing as t
@ -67,6 +72,7 @@ class Trino(Presto):
exp.ArraySum: lambda self, exp.ArraySum: lambda self,
e: f"REDUCE({self.sql(e, 'this')}, 0, (acc, x) -> acc + x, acc -> acc)", e: f"REDUCE({self.sql(e, 'this')}, 0, (acc, x) -> acc + x, acc -> acc)",
exp.ArrayUniqueAgg: lambda self, e: f"ARRAY_AGG(DISTINCT {self.sql(e, 'this')})", exp.ArrayUniqueAgg: lambda self, e: f"ARRAY_AGG(DISTINCT {self.sql(e, 'this')})",
exp.GroupConcat: lambda self, e: groupconcat_sql(self, e, on_overflow=True),
exp.LocationProperty: lambda self, e: self.property_sql(e), exp.LocationProperty: lambda self, e: self.property_sql(e),
exp.Merge: merge_without_target_sql, exp.Merge: merge_without_target_sql,
exp.TimeStrToTime: lambda self, e: timestrtotime_sql(self, e, include_precision=True), exp.TimeStrToTime: lambda self, e: timestrtotime_sql(self, e, include_precision=True),
@ -98,17 +104,3 @@ class Trino(Presto):
expression.this, expression.this,
json_path + option + quote + on_condition, json_path + option + quote + on_condition,
) )
def groupconcat_sql(self, expression: exp.GroupConcat) -> str:
this = expression.this
separator = expression.args.get("separator") or exp.Literal.string(",")
if isinstance(this, exp.Order):
if this.this:
this = this.this.pop()
on_overflow = self.sql(expression, "on_overflow")
on_overflow = f" ON OVERFLOW {on_overflow}" if on_overflow else ""
return f"LISTAGG({self.format_args(this, separator)}{on_overflow}) WITHIN GROUP ({self.sql(expression.this).lstrip()})"
return super().groupconcat_sql(expression)

14
sqlglot/expressions.py
View file

@ -3025,6 +3025,10 @@ class StabilityProperty(Property):
arg_types = {"this": True} arg_types = {"this": True}
class StorageHandlerProperty(Property):
arg_types = {"this": True}
class TemporaryProperty(Property): class TemporaryProperty(Property):
arg_types = {"this": False} arg_types = {"this": False}
@ -3093,6 +3097,10 @@ class IncludeProperty(Property):
arg_types = {"this": True, "alias": False, "column_def": False} arg_types = {"this": True, "alias": False, "column_def": False}
class ForceProperty(Property):
arg_types = {}
class Properties(Expression): class Properties(Expression):
arg_types = {"expressions": True} arg_types = {"expressions": True}
@ -3251,6 +3259,11 @@ class HistoricalData(Expression):
arg_types = {"this": True, "kind": True, "expression": True} arg_types = {"this": True, "kind": True, "expression": True}
# https://docs.snowflake.com/en/sql-reference/sql/put
class Put(Expression):
arg_types = {"this": True, "target": True, "properties": False}
class Table(Expression): class Table(Expression):
arg_types = { arg_types = {
"this": False, "this": False,
@ -4401,6 +4414,7 @@ class DataType(Expression):
BIGSERIAL = auto() BIGSERIAL = auto()
BINARY = auto() BINARY = auto()
BIT = auto() BIT = auto()
BLOB = auto()
BOOLEAN = auto() BOOLEAN = auto()
BPCHAR = auto() BPCHAR = auto()
CHAR = auto() CHAR = auto()

11
sqlglot/generator.py
View file

@ -210,6 +210,7 @@ class Generator(metaclass=_Generator):
exp.WithProcedureOptions: lambda self, e: f"WITH {self.expressions(e, flat=True)}", exp.WithProcedureOptions: lambda self, e: f"WITH {self.expressions(e, flat=True)}",
exp.WithSchemaBindingProperty: lambda self, e: f"WITH SCHEMA {self.sql(e, 'this')}", exp.WithSchemaBindingProperty: lambda self, e: f"WITH SCHEMA {self.sql(e, 'this')}",
exp.WithOperator: lambda self, e: f"{self.sql(e, 'this')} WITH {self.sql(e, 'op')}", exp.WithOperator: lambda self, e: f"{self.sql(e, 'this')} WITH {self.sql(e, 'op')}",
exp.ForceProperty: lambda *_: "FORCE",
} }
# Whether null ordering is supported in order by # Whether null ordering is supported in order by
@ -467,6 +468,7 @@ class Generator(metaclass=_Generator):
exp.DataType.Type.MEDIUMTEXT: "TEXT", exp.DataType.Type.MEDIUMTEXT: "TEXT",
exp.DataType.Type.LONGTEXT: "TEXT", exp.DataType.Type.LONGTEXT: "TEXT",
exp.DataType.Type.TINYTEXT: "TEXT", exp.DataType.Type.TINYTEXT: "TEXT",
exp.DataType.Type.BLOB: "VARBINARY",
exp.DataType.Type.MEDIUMBLOB: "BLOB", exp.DataType.Type.MEDIUMBLOB: "BLOB",
exp.DataType.Type.LONGBLOB: "BLOB", exp.DataType.Type.LONGBLOB: "BLOB",
exp.DataType.Type.TINYBLOB: "BLOB", exp.DataType.Type.TINYBLOB: "BLOB",
@ -584,6 +586,7 @@ class Generator(metaclass=_Generator):
exp.SqlReadWriteProperty: exp.Properties.Location.POST_SCHEMA, exp.SqlReadWriteProperty: exp.Properties.Location.POST_SCHEMA,
exp.SqlSecurityProperty: exp.Properties.Location.POST_CREATE, exp.SqlSecurityProperty: exp.Properties.Location.POST_CREATE,
exp.StabilityProperty: exp.Properties.Location.POST_SCHEMA, exp.StabilityProperty: exp.Properties.Location.POST_SCHEMA,
exp.StorageHandlerProperty: exp.Properties.Location.POST_SCHEMA,
exp.StreamingTableProperty: exp.Properties.Location.POST_CREATE, exp.StreamingTableProperty: exp.Properties.Location.POST_CREATE,
exp.StrictProperty: exp.Properties.Location.POST_SCHEMA, exp.StrictProperty: exp.Properties.Location.POST_SCHEMA,
exp.Tags: exp.Properties.Location.POST_WITH, exp.Tags: exp.Properties.Location.POST_WITH,
@ -600,6 +603,7 @@ class Generator(metaclass=_Generator):
exp.WithProcedureOptions: exp.Properties.Location.POST_SCHEMA, exp.WithProcedureOptions: exp.Properties.Location.POST_SCHEMA,
exp.WithSchemaBindingProperty: exp.Properties.Location.POST_SCHEMA, exp.WithSchemaBindingProperty: exp.Properties.Location.POST_SCHEMA,
exp.WithSystemVersioningProperty: exp.Properties.Location.POST_SCHEMA, exp.WithSystemVersioningProperty: exp.Properties.Location.POST_SCHEMA,
exp.ForceProperty: exp.Properties.Location.POST_CREATE,
} }
# Keywords that can't be used as unquoted identifier names # Keywords that can't be used as unquoted identifier names
@ -4846,3 +4850,10 @@ class Generator(metaclass=_Generator):
def show_sql(self, expression: exp.Show) -> str: def show_sql(self, expression: exp.Show) -> str:
self.unsupported("Unsupported SHOW statement") self.unsupported("Unsupported SHOW statement")
return "" return ""
def put_sql(self, expression: exp.Put) -> str:
props = expression.args.get("properties")
props_sql = self.properties(props, prefix=" ", sep=" ", wrapped=False) if props else ""
this = self.sql(expression, "this")
target = self.sql(expression, "target")
return f"PUT {this} {target}{props_sql}"

31
sqlglot/parser.py
View file

@ -327,6 +327,7 @@ class Parser(metaclass=_Parser):
TokenType.TEXT, TokenType.TEXT,
TokenType.MEDIUMTEXT, TokenType.MEDIUMTEXT,
TokenType.LONGTEXT, TokenType.LONGTEXT,
TokenType.BLOB,
TokenType.MEDIUMBLOB, TokenType.MEDIUMBLOB,
TokenType.LONGBLOB, TokenType.LONGBLOB,
TokenType.BINARY, TokenType.BINARY,
@ -527,6 +528,7 @@ class Parser(metaclass=_Parser):
TokenType.PERCENT, TokenType.PERCENT,
TokenType.PIVOT, TokenType.PIVOT,
TokenType.PRAGMA, TokenType.PRAGMA,
TokenType.PUT,
TokenType.RANGE, TokenType.RANGE,
TokenType.RECURSIVE, TokenType.RECURSIVE,
TokenType.REFERENCES, TokenType.REFERENCES,
@ -1290,6 +1292,10 @@ class Parser(metaclass=_Parser):
"SIMPLE", "SIMPLE",
), ),
"INITIALLY": ("DEFERRED", "IMMEDIATE"), "INITIALLY": ("DEFERRED", "IMMEDIATE"),
"USING": (
"BTREE",
"HASH",
),
**dict.fromkeys(("DEFERRABLE", "NORELY"), tuple()), **dict.fromkeys(("DEFERRABLE", "NORELY"), tuple()),
} }
@ -2119,9 +2125,11 @@ class Parser(metaclass=_Parser):
return self.expression(exp.Property, this=key, value=value) return self.expression(exp.Property, this=key, value=value)
def _parse_stored(self) -> exp.FileFormatProperty: def _parse_stored(self) -> t.Union[exp.FileFormatProperty, exp.StorageHandlerProperty]:
self._match(TokenType.ALIAS) if self._match_text_seq("BY"):
return self.expression(exp.StorageHandlerProperty, this=self._parse_var_or_string())
self._match(TokenType.ALIAS)
input_format = self._parse_string() if self._match_text_seq("INPUTFORMAT") else None input_format = self._parse_string() if self._match_text_seq("INPUTFORMAT") else None
output_format = self._parse_string() if self._match_text_seq("OUTPUTFORMAT") else None output_format = self._parse_string() if self._match_text_seq("OUTPUTFORMAT") else None
@ -2129,7 +2137,9 @@ class Parser(metaclass=_Parser):
exp.FileFormatProperty, exp.FileFormatProperty,
this=( this=(
self.expression( self.expression(
exp.InputOutputFormat, input_format=input_format, output_format=output_format exp.InputOutputFormat,
input_format=input_format,
output_format=output_format,
) )
if input_format or output_format if input_format or output_format
else self._parse_var_or_string() or self._parse_number() or self._parse_id_var() else self._parse_var_or_string() or self._parse_number() or self._parse_id_var()
@ -3599,7 +3609,10 @@ class Parser(metaclass=_Parser):
self, skip_join_token: bool = False, parse_bracket: bool = False self, skip_join_token: bool = False, parse_bracket: bool = False
) -> t.Optional[exp.Join]: ) -> t.Optional[exp.Join]:
if self._match(TokenType.COMMA): if self._match(TokenType.COMMA):
return self.expression(exp.Join, this=self._parse_table()) table = self._try_parse(self._parse_table)
if table:
return self.expression(exp.Join, this=table)
return None
index = self._index index = self._index
method, side, kind = self._parse_join_parts() method, side, kind = self._parse_join_parts()
@ -5335,6 +5348,8 @@ class Parser(metaclass=_Parser):
self.raise_error("Expected type") self.raise_error("Expected type")
elif op and self._curr: elif op and self._curr:
field = self._parse_column_reference() or self._parse_bracket() field = self._parse_column_reference() or self._parse_bracket()
if isinstance(field, exp.Column) and self._match(TokenType.DOT, advance=False):
field = self._parse_column_ops(field)
else: else:
field = self._parse_field(any_token=True, anonymous_func=True) field = self._parse_field(any_token=True, anonymous_func=True)
@ -5603,10 +5618,7 @@ class Parser(metaclass=_Parser):
def _parse_user_defined_function( def _parse_user_defined_function(
self, kind: t.Optional[TokenType] = None self, kind: t.Optional[TokenType] = None
) -> t.Optional[exp.Expression]: ) -> t.Optional[exp.Expression]:
this = self._parse_id_var() this = self._parse_table_parts(schema=True)
while self._match(TokenType.DOT):
this = self.expression(exp.Dot, this=this, expression=self._parse_id_var())
if not self._match(TokenType.L_PAREN): if not self._match(TokenType.L_PAREN):
return this return this
@ -6254,7 +6266,8 @@ class Parser(metaclass=_Parser):
if not self._match(TokenType.R_PAREN) and args: if not self._match(TokenType.R_PAREN) and args:
# postgres: STRING_AGG([DISTINCT] expression, separator [ORDER BY expression1 {ASC | DESC} [, ...]]) # postgres: STRING_AGG([DISTINCT] expression, separator [ORDER BY expression1 {ASC | DESC} [, ...]])
# bigquery: STRING_AGG([DISTINCT] expression [, separator] [ORDER BY key [{ASC | DESC}] [, ... ]] [LIMIT n]) # bigquery: STRING_AGG([DISTINCT] expression [, separator] [ORDER BY key [{ASC | DESC}] [, ... ]] [LIMIT n])
args[-1] = self._parse_limit(this=self._parse_order(this=args[-1])) # The order is parsed through `this` as a canonicalization for WITHIN GROUPs
args[0] = self._parse_limit(this=self._parse_order(this=args[0]))
return self.expression(exp.GroupConcat, this=args[0], separator=seq_get(args, 1)) return self.expression(exp.GroupConcat, this=args[0], separator=seq_get(args, 1))
# Checks if we can parse an order clause: WITHIN GROUP (ORDER BY <order_by_expression_list> [ASC | DESC]). # Checks if we can parse an order clause: WITHIN GROUP (ORDER BY <order_by_expression_list> [ASC | DESC]).

4
sqlglot/tokens.py
View file

@ -80,6 +80,8 @@ class TokenType(AutoName):
XOR = auto() XOR = auto()
DSTAR = auto() DSTAR = auto()
URI_START = auto()
BLOCK_START = auto() BLOCK_START = auto()
BLOCK_END = auto() BLOCK_END = auto()
@ -140,6 +142,7 @@ class TokenType(AutoName):
TEXT = auto() TEXT = auto()
MEDIUMTEXT = auto() MEDIUMTEXT = auto()
LONGTEXT = auto() LONGTEXT = auto()
BLOB = auto()
MEDIUMBLOB = auto() MEDIUMBLOB = auto()
LONGBLOB = auto() LONGBLOB = auto()
TINYBLOB = auto() TINYBLOB = auto()
@ -355,6 +358,7 @@ class TokenType(AutoName):
PROCEDURE = auto() PROCEDURE = auto()
PROPERTIES = auto() PROPERTIES = auto()
PSEUDO_TYPE = auto() PSEUDO_TYPE = auto()
PUT = auto()
QUALIFY = auto() QUALIFY = auto()
QUOTE = auto() QUOTE = auto()
RANGE = auto() RANGE = auto()

27
tests/dialects/test_bigquery.py
View file

@ -142,7 +142,6 @@ LANGUAGE js AS
self.validate_identity("SELECT test.Unknown FROM test") self.validate_identity("SELECT test.Unknown FROM test")
self.validate_identity(r"SELECT '\n\r\a\v\f\t'") self.validate_identity(r"SELECT '\n\r\a\v\f\t'")
self.validate_identity("SELECT * FROM tbl FOR SYSTEM_TIME AS OF z") self.validate_identity("SELECT * FROM tbl FOR SYSTEM_TIME AS OF z")
self.validate_identity("STRING_AGG(DISTINCT a ORDER BY b DESC, c DESC LIMIT 10)")
self.validate_identity("SELECT PARSE_TIMESTAMP('%c', 'Thu Dec 25 07:30:00 2008', 'UTC')") self.validate_identity("SELECT PARSE_TIMESTAMP('%c', 'Thu Dec 25 07:30:00 2008', 'UTC')")
self.validate_identity("SELECT ANY_VALUE(fruit HAVING MAX sold) FROM fruits") self.validate_identity("SELECT ANY_VALUE(fruit HAVING MAX sold) FROM fruits")
self.validate_identity("SELECT ANY_VALUE(fruit HAVING MIN sold) FROM fruits") self.validate_identity("SELECT ANY_VALUE(fruit HAVING MIN sold) FROM fruits")
@ -150,10 +149,6 @@ LANGUAGE js AS
self.validate_identity("SELECT CAST(CURRENT_DATE AS STRING FORMAT 'DAY') AS current_day") self.validate_identity("SELECT CAST(CURRENT_DATE AS STRING FORMAT 'DAY') AS current_day")
self.validate_identity("SAFE_CAST(encrypted_value AS STRING FORMAT 'BASE64')") self.validate_identity("SAFE_CAST(encrypted_value AS STRING FORMAT 'BASE64')")
self.validate_identity("CAST(encrypted_value AS STRING FORMAT 'BASE64')") self.validate_identity("CAST(encrypted_value AS STRING FORMAT 'BASE64')")
self.validate_identity("STRING_AGG(a)")
self.validate_identity("STRING_AGG(a, ' & ')")
self.validate_identity("STRING_AGG(DISTINCT a, ' & ')")
self.validate_identity("STRING_AGG(a, ' & ' ORDER BY LENGTH(a))")
self.validate_identity("DATE(2016, 12, 25)") self.validate_identity("DATE(2016, 12, 25)")
self.validate_identity("DATE(CAST('2016-12-25 23:59:59' AS DATETIME))") self.validate_identity("DATE(CAST('2016-12-25 23:59:59' AS DATETIME))")
self.validate_identity("SELECT foo IN UNNEST(bar) AS bla") self.validate_identity("SELECT foo IN UNNEST(bar) AS bla")
@ -1844,12 +1839,6 @@ WHERE
"CREATE TEMPORARY FUNCTION string_length_0(strings ARRAY<STRING>) RETURNS FLOAT64 LANGUAGE js OPTIONS (library=['gs://ibis-testing-libraries/lodash.min.js']) AS '\\'use strict\\'; function string_length(strings) { return _.sum(_.map(strings, ((x) => x.length))); } return string_length(strings);'", "CREATE TEMPORARY FUNCTION string_length_0(strings ARRAY<STRING>) RETURNS FLOAT64 LANGUAGE js OPTIONS (library=['gs://ibis-testing-libraries/lodash.min.js']) AS '\\'use strict\\'; function string_length(strings) { return _.sum(_.map(strings, ((x) => x.length))); } return string_length(strings);'",
) )
def test_group_concat(self):
self.validate_all(
"SELECT a, GROUP_CONCAT(b) FROM table GROUP BY a",
write={"bigquery": "SELECT a, STRING_AGG(b) FROM table GROUP BY a"},
)
def test_remove_precision_parameterized_types(self): def test_remove_precision_parameterized_types(self):
self.validate_identity("CREATE TABLE test (a NUMERIC(10, 2))") self.validate_identity("CREATE TABLE test (a NUMERIC(10, 2))")
self.validate_identity( self.validate_identity(
@ -2347,3 +2336,19 @@ OPTIONS (
"snowflake": "SELECT TO_CHAR(CAST(CAST('2050-12-25 15:30:55+00' AS TIMESTAMPTZ) AS TIMESTAMP), 'mon-DD-yyyy')", "snowflake": "SELECT TO_CHAR(CAST(CAST('2050-12-25 15:30:55+00' AS TIMESTAMPTZ) AS TIMESTAMP), 'mon-DD-yyyy')",
}, },
) )
def test_string_agg(self):
self.validate_identity(
"SELECT a, GROUP_CONCAT(b) FROM table GROUP BY a",
"SELECT a, STRING_AGG(b, ',') FROM table GROUP BY a",
)
self.validate_identity("STRING_AGG(a, ' & ')")
self.validate_identity("STRING_AGG(DISTINCT a, ' & ')")
self.validate_identity("STRING_AGG(a, ' & ' ORDER BY LENGTH(a))")
self.validate_identity("STRING_AGG(a)", "STRING_AGG(a, ',')")
self.validate_identity(
"STRING_AGG(DISTINCT a ORDER BY b DESC, c DESC LIMIT 10)",
"STRING_AGG(DISTINCT a, ',' ORDER BY b DESC, c DESC LIMIT 10)",
)

1
tests/dialects/test_hive.py
View file

@ -173,6 +173,7 @@ class TestHive(Validator):
self.validate_identity( self.validate_identity(
"""CREATE EXTERNAL TABLE `my_table` (`a7` ARRAY<DATE>) ROW FORMAT SERDE 'a' STORED AS INPUTFORMAT 'b' OUTPUTFORMAT 'c' LOCATION 'd' TBLPROPERTIES ('e'='f')""" """CREATE EXTERNAL TABLE `my_table` (`a7` ARRAY<DATE>) ROW FORMAT SERDE 'a' STORED AS INPUTFORMAT 'b' OUTPUTFORMAT 'c' LOCATION 'd' TBLPROPERTIES ('e'='f')"""
) )
self.validate_identity("CREATE EXTERNAL TABLE X (y INT) STORED BY 'x'")
self.validate_identity("ALTER VIEW v1 AS SELECT x, UPPER(s) AS s FROM t2") self.validate_identity("ALTER VIEW v1 AS SELECT x, UPPER(s) AS s FROM t2")
self.validate_identity("ALTER VIEW v1 (c1, c2) AS SELECT x, UPPER(s) AS s FROM t2") self.validate_identity("ALTER VIEW v1 (c1, c2) AS SELECT x, UPPER(s) AS s FROM t2")
self.validate_identity( self.validate_identity(

22
tests/dialects/test_mysql.py
View file

@ -83,6 +83,12 @@ class TestMySQL(Validator):
self.validate_identity( self.validate_identity(
"CREATE OR REPLACE VIEW my_view AS SELECT column1 AS `boo`, column2 AS `foo` FROM my_table WHERE column3 = 'some_value' UNION SELECT q.* FROM fruits_table, JSON_TABLE(Fruits, '$[*]' COLUMNS(id VARCHAR(255) PATH '$.$id', value VARCHAR(255) PATH '$.value')) AS q", "CREATE OR REPLACE VIEW my_view AS SELECT column1 AS `boo`, column2 AS `foo` FROM my_table WHERE column3 = 'some_value' UNION SELECT q.* FROM fruits_table, JSON_TABLE(Fruits, '$[*]' COLUMNS(id VARCHAR(255) PATH '$.$id', value VARCHAR(255) PATH '$.value')) AS q",
) )
self.validate_identity(
"CREATE TABLE test_table (id INT AUTO_INCREMENT, PRIMARY KEY (id) USING BTREE)"
)
self.validate_identity(
"CREATE TABLE test_table (id INT AUTO_INCREMENT, PRIMARY KEY (id) USING HASH)"
)
self.validate_identity( self.validate_identity(
"/*left*/ EXPLAIN SELECT /*hint*/ col FROM t1 /*right*/", "/*left*/ EXPLAIN SELECT /*hint*/ col FROM t1 /*right*/",
"/* left */ DESCRIBE /* hint */ SELECT col FROM t1 /* right */", "/* left */ DESCRIBE /* hint */ SELECT col FROM t1 /* right */",
@ -340,6 +346,22 @@ class TestMySQL(Validator):
"tsql": "CHAR(10)", "tsql": "CHAR(10)",
}, },
) )
self.validate_identity("CREATE TABLE t (foo VARBINARY(5))")
self.validate_all(
"CREATE TABLE t (foo BLOB)",
write={
"mysql": "CREATE TABLE t (foo BLOB)",
"oracle": "CREATE TABLE t (foo BLOB)",
"postgres": "CREATE TABLE t (foo BYTEA)",
"tsql": "CREATE TABLE t (foo VARBINARY)",
"sqlite": "CREATE TABLE t (foo BLOB)",
"duckdb": "CREATE TABLE t (foo VARBINARY)",
"hive": "CREATE TABLE t (foo BINARY)",
"bigquery": "CREATE TABLE t (foo BYTES)",
"redshift": "CREATE TABLE t (foo VARBYTE)",
"clickhouse": "CREATE TABLE t (foo Nullable(String))",
},
)
def test_escape(self): def test_escape(self):
self.validate_identity("""'"abc"'""") self.validate_identity("""'"abc"'""")

1
tests/dialects/test_oracle.py
View file

@ -319,6 +319,7 @@ class TestOracle(Validator):
"tsql": "SELECT * FROM t ORDER BY (SELECT NULL) OFFSET 0 ROWS FETCH FIRST 10 ROWS ONLY", "tsql": "SELECT * FROM t ORDER BY (SELECT NULL) OFFSET 0 ROWS FETCH FIRST 10 ROWS ONLY",
}, },
) )
self.validate_identity("CREATE OR REPLACE FORCE VIEW foo1.foo2")
def test_join_marker(self): def test_join_marker(self):
self.validate_identity("SELECT e1.x, e2.x FROM e e1, e e2 WHERE e1.y (+) = e2.y") self.validate_identity("SELECT e1.x, e2.x FROM e e1, e e2 WHERE e1.y (+) = e2.y")

44
tests/dialects/test_postgres.py
View file

@ -345,6 +345,28 @@ class TestPostgres(Validator):
"CAST(x AS INT8)", "CAST(x AS INT8)",
"CAST(x AS BIGINT)", "CAST(x AS BIGINT)",
) )
self.validate_identity(
"""
WITH
json_data AS (SELECT '{"field_id": [1, 2, 3]}'::JSON AS data),
field_ids AS (SELECT 'field_id' AS field_id)
SELECT
JSON_ARRAY_ELEMENTS(json_data.data -> field_ids.field_id) AS element
FROM json_data, field_ids
""",
"""WITH json_data AS (
SELECT
CAST('{"field_id": [1, 2, 3]}' AS JSON) AS data
), field_ids AS (
SELECT
'field_id' AS field_id
)
SELECT
JSON_ARRAY_ELEMENTS(JSON_EXTRACT_PATH(json_data.data, field_ids.field_id)) AS element
FROM json_data, field_ids""",
pretty=True,
)
self.validate_all( self.validate_all(
"SELECT ARRAY[]::INT[] AS foo", "SELECT ARRAY[]::INT[] AS foo",
@ -1368,3 +1390,25 @@ CROSS JOIN JSON_ARRAY_ELEMENTS(CAST(JSON_EXTRACT_PATH(tbox, 'boxes') AS JSON)) A
self.validate_identity( self.validate_identity(
"WITH RECURSIVE search_graph(id, link, data, depth) AS (SELECT g.id, g.link, g.data, 1 FROM graph AS g UNION ALL SELECT g.id, g.link, g.data, sg.depth + 1 FROM graph AS g, search_graph AS sg WHERE g.id = sg.link) CYCLE id SET is_cycle USING path SELECT * FROM search_graph" "WITH RECURSIVE search_graph(id, link, data, depth) AS (SELECT g.id, g.link, g.data, 1 FROM graph AS g UNION ALL SELECT g.id, g.link, g.data, sg.depth + 1 FROM graph AS g, search_graph AS sg WHERE g.id = sg.link) CYCLE id SET is_cycle USING path SELECT * FROM search_graph"
) )
def test_json_extract(self):
for arrow_op in ("->", "->>"):
with self.subTest(f"Ensure {arrow_op} operator roundtrips int values as subscripts"):
self.validate_all(
f"SELECT foo {arrow_op} 1",
write={
"postgres": f"SELECT foo {arrow_op} 1",
"duckdb": f"SELECT foo {arrow_op} '$[1]'",
},
)
with self.subTest(
f"Ensure {arrow_op} operator roundtrips string values that represent integers as keys"
):
self.validate_all(
f"SELECT foo {arrow_op} '12'",
write={
"postgres": f"SELECT foo {arrow_op} '12'",
"clickhouse": "SELECT JSONExtractString(foo, '12')",
},
)

4
tests/dialects/test_redshift.py
View file

@ -42,8 +42,8 @@ class TestRedshift(Validator):
"duckdb": "STRING_AGG(sellerid, ', ')", "duckdb": "STRING_AGG(sellerid, ', ')",
}, },
write={ write={
# GROUP_CONCAT and STRING_AGG are aliases in DuckDB # GROUP_CONCAT, LISTAGG and STRING_AGG are aliases in DuckDB
"duckdb": "GROUP_CONCAT(sellerid, ', ')", "duckdb": "LISTAGG(sellerid, ', ')",
"redshift": "LISTAGG(sellerid, ', ')", "redshift": "LISTAGG(sellerid, ', ')",
}, },
) )

87
tests/dialects/test_snowflake.py
View file

@ -52,7 +52,6 @@ class TestSnowflake(Validator):
self.validate_identity("SELECT OBJECT_CONSTRUCT()") self.validate_identity("SELECT OBJECT_CONSTRUCT()")
self.validate_identity("SELECT DAYOFMONTH(CURRENT_TIMESTAMP())") self.validate_identity("SELECT DAYOFMONTH(CURRENT_TIMESTAMP())")
self.validate_identity("SELECT DAYOFYEAR(CURRENT_TIMESTAMP())") self.validate_identity("SELECT DAYOFYEAR(CURRENT_TIMESTAMP())")
self.validate_identity("LISTAGG(data['some_field'], ',')")
self.validate_identity("WEEKOFYEAR(tstamp)") self.validate_identity("WEEKOFYEAR(tstamp)")
self.validate_identity("SELECT QUARTER(CURRENT_TIMESTAMP())") self.validate_identity("SELECT QUARTER(CURRENT_TIMESTAMP())")
self.validate_identity("SELECT SUM(amount) FROM mytable GROUP BY ALL") self.validate_identity("SELECT SUM(amount) FROM mytable GROUP BY ALL")
@ -107,6 +106,10 @@ class TestSnowflake(Validator):
self.validate_identity( self.validate_identity(
"""SELECT TO_TIMESTAMP('2025-01-16T14:45:30.123+0500', 'yyyy-mm-DD"T"hh24:mi:ss.ff3TZHTZM')""" """SELECT TO_TIMESTAMP('2025-01-16T14:45:30.123+0500', 'yyyy-mm-DD"T"hh24:mi:ss.ff3TZHTZM')"""
) )
self.validate_identity(
"SELECT 1 put",
"SELECT 1 AS put",
)
self.validate_identity( self.validate_identity(
"WITH t (SELECT 1 AS c) SELECT c FROM t", "WITH t (SELECT 1 AS c) SELECT c FROM t",
"WITH t AS (SELECT 1 AS c) SELECT c FROM t", "WITH t AS (SELECT 1 AS c) SELECT c FROM t",
@ -296,6 +299,27 @@ class TestSnowflake(Validator):
"SELECT * RENAME (a AS b), c AS d FROM xxx", "SELECT * RENAME (a AS b), c AS d FROM xxx",
) )
# Support for optional trailing commas after tables in from clause
self.validate_identity(
"SELECT * FROM xxx, yyy, zzz,",
"SELECT * FROM xxx, yyy, zzz",
)
self.validate_identity(
"SELECT * FROM xxx, yyy, zzz, WHERE foo = bar",
"SELECT * FROM xxx, yyy, zzz WHERE foo = bar",
)
self.validate_identity(
"SELECT * FROM xxx, yyy, zzz",
"SELECT * FROM xxx, yyy, zzz",
)
self.validate_all(
"CREATE TABLE test_table (id NUMERIC NOT NULL AUTOINCREMENT)",
write={
"duckdb": "CREATE TABLE test_table (id DECIMAL(38, 0) NOT NULL)",
"snowflake": "CREATE TABLE test_table (id DECIMAL(38, 0) NOT NULL AUTOINCREMENT)",
},
)
self.validate_all( self.validate_all(
"SELECT TO_TIMESTAMP('2025-01-16 14:45:30.123', 'yyyy-mm-DD hh24:mi:ss.ff6')", "SELECT TO_TIMESTAMP('2025-01-16 14:45:30.123', 'yyyy-mm-DD hh24:mi:ss.ff6')",
write={ write={
@ -852,13 +876,6 @@ class TestSnowflake(Validator):
"snowflake": "CASE WHEN x = a OR (x IS NULL AND a IS NULL) THEN b WHEN x = c OR (x IS NULL AND c IS NULL) THEN d ELSE e END", "snowflake": "CASE WHEN x = a OR (x IS NULL AND a IS NULL) THEN b WHEN x = c OR (x IS NULL AND c IS NULL) THEN d ELSE e END",
}, },
) )
self.validate_all(
"SELECT LISTAGG(col1, ', ') WITHIN GROUP (ORDER BY col2) FROM t",
write={
"duckdb": "SELECT GROUP_CONCAT(col1, ', ' ORDER BY col2) FROM t",
"snowflake": "SELECT LISTAGG(col1, ', ') WITHIN GROUP (ORDER BY col2) FROM t",
},
)
self.validate_all( self.validate_all(
"SELECT APPROX_PERCENTILE(a, 0.5) FROM t", "SELECT APPROX_PERCENTILE(a, 0.5) FROM t",
read={ read={
@ -2369,6 +2386,43 @@ SINGLE = TRUE""",
"""COPY INTO 's3://example/contacts.csv' FROM "db"."tbl" STORAGE_INTEGRATION = "PROD_S3_SIDETRADE_INTEGRATION" FILE_FORMAT = (FORMAT_NAME="my_csv_format" TYPE=CSV COMPRESSION=NONE NULL_IF=('') FIELD_OPTIONALLY_ENCLOSED_BY='"') MATCH_BY_COLUMN_NAME = CASE_SENSITIVE OVERWRITE = TRUE SINGLE = TRUE INCLUDE_METADATA = ("col1" = "METADATA$START_SCAN_TIME")""", """COPY INTO 's3://example/contacts.csv' FROM "db"."tbl" STORAGE_INTEGRATION = "PROD_S3_SIDETRADE_INTEGRATION" FILE_FORMAT = (FORMAT_NAME="my_csv_format" TYPE=CSV COMPRESSION=NONE NULL_IF=('') FIELD_OPTIONALLY_ENCLOSED_BY='"') MATCH_BY_COLUMN_NAME = CASE_SENSITIVE OVERWRITE = TRUE SINGLE = TRUE INCLUDE_METADATA = ("col1" = "METADATA$START_SCAN_TIME")""",
) )
def test_put_to_stage(self):
# PUT with file path and stage ref containing spaces (wrapped in single quotes)
ast = parse_one("PUT 'file://my file.txt' '@s1/my folder'", read="snowflake")
self.assertIsInstance(ast, exp.Put)
self.assertEqual(ast.this, exp.Literal(this="file://my file.txt", is_string=True))
self.assertEqual(ast.args["target"], exp.Var(this="@s1/my folder"))
# expression with additional properties
ast = parse_one(
"PUT 'file:///tmp/my.txt' @stage1/folder PARALLEL = 1 AUTO_COMPRESS=false source_compression=gzip OVERWRITE=TRUE",
read="snowflake",
)
self.assertIsInstance(ast, exp.Put)
self.assertEqual(ast.this, exp.Literal(this="file:///tmp/my.txt", is_string=True))
self.assertEqual(ast.args["target"], exp.Var(this="@stage1/folder"))
properties = ast.args.get("properties")
props_dict = {prop.this.this: prop.args["value"].this for prop in properties.expressions}
self.assertEqual(
props_dict,
{
"PARALLEL": "1",
"AUTO_COMPRESS": False,
"source_compression": "gzip",
"OVERWRITE": True,
},
)
# validate identity for different args and properties
self.validate_identity("PUT 'file:///dir/tmp.csv' @s1/test")
# the unquoted URI variant is not fully supported yet
self.validate_identity("PUT file:///dir/tmp.csv @%table", check_command_warning=True)
self.validate_identity(
"PUT file:///dir/tmp.csv @s1/test PARALLEL=1 AUTO_COMPRESS=FALSE source_compression=gzip OVERWRITE=TRUE",
check_command_warning=True,
)
def test_querying_semi_structured_data(self): def test_querying_semi_structured_data(self):
self.validate_identity("SELECT $1") self.validate_identity("SELECT $1")
self.validate_identity("SELECT $1.elem") self.validate_identity("SELECT $1.elem")
@ -2450,3 +2504,20 @@ SINGLE = TRUE""",
"trino": "SELECT 1 ORDER BY 1 OFFSET 0", "trino": "SELECT 1 ORDER BY 1 OFFSET 0",
}, },
) )
def test_listagg(self):
self.validate_identity("LISTAGG(data['some_field'], ',')")
for distinct in ("", "DISTINCT "):
self.validate_all(
f"SELECT LISTAGG({distinct}col, '|SEPARATOR|') WITHIN GROUP (ORDER BY col2) FROM t",
read={
"trino": f"SELECT LISTAGG({distinct}col, '|SEPARATOR|') WITHIN GROUP (ORDER BY col2) FROM t",
"duckdb": f"SELECT LISTAGG({distinct}col, '|SEPARATOR|' ORDER BY col2) FROM t",
},
write={
"snowflake": f"SELECT LISTAGG({distinct}col, '|SEPARATOR|') WITHIN GROUP (ORDER BY col2) FROM t",
"trino": f"SELECT LISTAGG({distinct}col, '|SEPARATOR|') WITHIN GROUP (ORDER BY col2) FROM t",
"duckdb": f"SELECT LISTAGG({distinct}col, '|SEPARATOR|' ORDER BY col2) FROM t",
},
)

1
tests/fixtures/partial.sql vendored
View file

@ -2,7 +2,6 @@ SELECT a FROM
SELECT a FROM x WHERE SELECT a FROM x WHERE
SELECT a + SELECT a +
a * a *
SELECT a FROM x,
SELECT a FROM x GROUP BY SELECT a FROM x GROUP BY
WITH a AS (SELECT 1), b AS (SELECT 2) WITH a AS (SELECT 1), b AS (SELECT 2)
SELECT FROM x SELECT FROM x