dbcli/sqlglot
1
0
Fork 0
Code Issues Activity

Merging upstream version 26.8.0.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-03-04 07:54:36 +01:00
parent d551ab0954
commit 010433ad9a
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
61 changed files with 43883 additions and 41898 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

130
CHANGELOG.md
View file

@ -1,6 +1,134 @@
Changelog
=========
## [v26.7.0] - 2025-02-26
### :boom: BREAKING CHANGES
- due to [`466c839`](https://github.com/tobymao/sqlglot/commit/466c839c2cfc94b398dd619b738df165f2876cdb) - Remove extra MAP bracket and ARRAY wrap *(PR [#4712](https://github.com/tobymao/sqlglot/pull/4712) by [@VaggelisD](https://github.com/VaggelisD))*:
Remove extra MAP bracket and ARRAY wrap (#4712)
- due to [`79ab311`](https://github.com/tobymao/sqlglot/commit/79ab3116758c240786ab4353a26f1646e242a61b) - add generate_series table column alias *(PR [#4741](https://github.com/tobymao/sqlglot/pull/4741) by [@georgesittas](https://github.com/georgesittas))*:
add generate_series table column alias (#4741)
- due to [`66b3ea9`](https://github.com/tobymao/sqlglot/commit/66b3ea905af34cfedc961c68ba738ba90d16221d) - respect type nullability when casting in strtodate_sql *(PR [#4744](https://github.com/tobymao/sqlglot/pull/4744) by [@sleshJdev](https://github.com/sleshJdev))*:
respect type nullability when casting in strtodate_sql (#4744)
- due to [`91f47fe`](https://github.com/tobymao/sqlglot/commit/91f47fec2a8c727f7e4c93fa54b6f06a36a6b42f) - Support for exp.HexString in DuckDB/Presto/Trino *(PR [#4743](https://github.com/tobymao/sqlglot/pull/4743) by [@VaggelisD](https://github.com/VaggelisD))*:
Support for exp.HexString in DuckDB/Presto/Trino (#4743)
- due to [`0596176`](https://github.com/tobymao/sqlglot/commit/0596176dd59737f945624d6453259072917e2fee) - generate CASE expression instead of COUNT_IF for <v1.2 *(PR [#4755](https://github.com/tobymao/sqlglot/pull/4755) by [@georgesittas](https://github.com/georgesittas))*:
generate CASE expression instead of COUNT_IF for <v1.2 (#4755)
- due to [`01008a9`](https://github.com/tobymao/sqlglot/commit/01008a91144a20e830df3783ff04beba7d029fec) - enable transpilation of CURDATE fixes [#4758](https://github.com/tobymao/sqlglot/pull/4758) *(commit by [@georgesittas](https://github.com/georgesittas))*:
enable transpilation of CURDATE fixes #4758
- due to [`dc69192`](https://github.com/tobymao/sqlglot/commit/dc691923655d022f2a4b23e9df24a9d1518e7048) - column qualify not applying in an order by inside within group when it also the alias *(PR [#4780](https://github.com/tobymao/sqlglot/pull/4780) by [@ran-lakeway](https://github.com/ran-lakeway))*:
column qualify not applying in an order by inside within group when it also the alias (#4780)
- due to [`ae00c92`](https://github.com/tobymao/sqlglot/commit/ae00c9203197a436bdb81289580eba0e7fc374ec) - properly alias generate series *(PR [#4789](https://github.com/tobymao/sqlglot/pull/4789) by [@georgesittas](https://github.com/georgesittas))*:
properly alias generate series (#4789)
- due to [`14c3de9`](https://github.com/tobymao/sqlglot/commit/14c3de91072435a68dd8d0f024dff0a0fd236cb8) - parse unqualified names in DISTINCT ON (...) as identifiers *(PR [#4795](https://github.com/tobymao/sqlglot/pull/4795) by [@georgesittas](https://github.com/georgesittas))*:
parse unqualified names in DISTINCT ON (...) as identifiers (#4795)
- due to [`00c7f05`](https://github.com/tobymao/sqlglot/commit/00c7f05f348eeb6b60d4e70d7e29e564caf38f64) - do not coerce parameterized types *(PR [#4796](https://github.com/tobymao/sqlglot/pull/4796) by [@georgesittas](https://github.com/georgesittas))*:
do not coerce parameterized types (#4796)
- due to [`513fad4`](https://github.com/tobymao/sqlglot/commit/513fad47cd321fe8431e673351bdd5f0674d5af1) - show databases, functions, procedures and warehouses *(PR [#4787](https://github.com/tobymao/sqlglot/pull/4787) by [@tekumara](https://github.com/tekumara))*:
show databases, functions, procedures and warehouses (#4787)
- due to [`a9ae2d2`](https://github.com/tobymao/sqlglot/commit/a9ae2d229640f22134410ca5826c6f19df7ef523) - support TOP with PERCENT and WITH TIES *(PR [#4801](https://github.com/tobymao/sqlglot/pull/4801) by [@geooo109](https://github.com/geooo109))*:
support TOP with PERCENT and WITH TIES (#4801)
- due to [`e05983a`](https://github.com/tobymao/sqlglot/commit/e05983a62ee9b3689e8a277924f2b1e36744a2a9) - make hex literal tokenization more robust *(PR [#4802](https://github.com/tobymao/sqlglot/pull/4802) by [@georgesittas](https://github.com/georgesittas))*:
make hex literal tokenization more robust (#4802)
- due to [`e9c8eae`](https://github.com/tobymao/sqlglot/commit/e9c8eae060c05d02f6f65916d59f1792098679c8) - bump sqlglotrs to 0.4.0 *(commit by [@georgesittas](https://github.com/georgesittas))*:
bump sqlglotrs to 0.4.0
### :sparkles: New Features
- [`5c59943`](https://github.com/tobymao/sqlglot/commit/5c599434fbe4d4f1059196b1530d02d7d204d3a9) - **mysql**: add unsigned double type *(PR [#4734](https://github.com/tobymao/sqlglot/pull/4734) by [@GabrielVSMachado](https://github.com/GabrielVSMachado))*
- [`31b2139`](https://github.com/tobymao/sqlglot/commit/31b21391cc521cf15a098c335d4202378ad96c0b) - **duckdb**: transpile JSONB into JSON *(PR [#4753](https://github.com/tobymao/sqlglot/pull/4753) by [@georgesittas](https://github.com/georgesittas))*
- [`0596176`](https://github.com/tobymao/sqlglot/commit/0596176dd59737f945624d6453259072917e2fee) - **duckdb**: generate CASE expression instead of COUNT_IF for <v1.2 *(PR [#4755](https://github.com/tobymao/sqlglot/pull/4755) by [@georgesittas](https://github.com/georgesittas))*
- [`2b3ec0f`](https://github.com/tobymao/sqlglot/commit/2b3ec0fbd6bd84dd893c244892fc1a63e2eab7bd) - **snowflake**: add support for USE SECONDARY ROLES *(PR [#4761](https://github.com/tobymao/sqlglot/pull/4761) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#4760](https://github.com/tobymao/sqlglot/issues/4760) opened by [@merlindso](https://github.com/merlindso)*
- [`2079bdb`](https://github.com/tobymao/sqlglot/commit/2079bdbb5d72d7fe02019a9ba6d1684403e30b00) - **postgres**: Add support for WITH RECURSIVE search options *(PR [#4773](https://github.com/tobymao/sqlglot/pull/4773) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *addresses issue [#4763](https://github.com/tobymao/sqlglot/issues/4763) opened by [@psanjeev-cellino](https://github.com/psanjeev-cellino)*
- [`70eefc6`](https://github.com/tobymao/sqlglot/commit/70eefc694c20600cdd99d682393cae81812d0b00) - **duckdb**: add support for underscored numbers fixes [#4777](https://github.com/tobymao/sqlglot/pull/4777) *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`f6be1d4`](https://github.com/tobymao/sqlglot/commit/f6be1d4b2c34ea4e3e98c3ae5e272e2f2b1912f3) - **clickhouse**: add support for dynamic json casting *(PR [#4784](https://github.com/tobymao/sqlglot/pull/4784) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#4776](https://github.com/tobymao/sqlglot/issues/4776) opened by [@pkit](https://github.com/pkit)*
- [`513fad4`](https://github.com/tobymao/sqlglot/commit/513fad47cd321fe8431e673351bdd5f0674d5af1) - **snowflake**: show databases, functions, procedures and warehouses *(PR [#4787](https://github.com/tobymao/sqlglot/pull/4787) by [@tekumara](https://github.com/tekumara))*
- [`575c979`](https://github.com/tobymao/sqlglot/commit/575c979b77831bd356cc7e9b39e78fcca2b344be) - **clickhouse**: support `GLOBAL NOT IN` *(PR [#4798](https://github.com/tobymao/sqlglot/pull/4798) by [@hdhoang](https://github.com/hdhoang))*
- [`856faa9`](https://github.com/tobymao/sqlglot/commit/856faa9773dea3b214571932c7d65e5773c5dc76) - **mysql**: add support for `ALTER INDEX` *(PR [#4800](https://github.com/tobymao/sqlglot/pull/4800) by [@burnison](https://github.com/burnison))*
### :bug: Bug Fixes
- [`7cad7f0`](https://github.com/tobymao/sqlglot/commit/7cad7f06d676bf230238bf08aea91701e1fa1d95) - Add additional allowed tokens for parsing aggregate functions in DDL *(PR [#4727](https://github.com/tobymao/sqlglot/pull/4727) by [@dorranh](https://github.com/dorranh))*
- :arrow_lower_right: *fixes issue [#4723](https://github.com/tobymao/sqlglot/issues/4723) opened by [@dorranh](https://github.com/dorranh)*
- [`466c839`](https://github.com/tobymao/sqlglot/commit/466c839c2cfc94b398dd619b738df165f2876cdb) - **duckdb**: Remove extra MAP bracket and ARRAY wrap *(PR [#4712](https://github.com/tobymao/sqlglot/pull/4712) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4710](https://github.com/tobymao/sqlglot/issues/4710) opened by [@eakmanrq](https://github.com/eakmanrq)*
- [`2c7dbe2`](https://github.com/tobymao/sqlglot/commit/2c7dbe2f84fcba044881776ebdeed6ee980fe0dd) - **duckdb**: Fix INTERVAL roundtrip of DATE_ADD *(PR [#4732](https://github.com/tobymao/sqlglot/pull/4732) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4728](https://github.com/tobymao/sqlglot/issues/4728) opened by [@whao89](https://github.com/whao89)*
- [`cbad48d`](https://github.com/tobymao/sqlglot/commit/cbad48d4102412ea3fd20272cf75974a8c53bb34) - infinite traversal in eliminate_qualify closes [#4735](https://github.com/tobymao/sqlglot/pull/4735) *(commit by [@tobymao](https://github.com/tobymao))*
- [`79ab311`](https://github.com/tobymao/sqlglot/commit/79ab3116758c240786ab4353a26f1646e242a61b) - **clickhouse**: add generate_series table column alias *(PR [#4741](https://github.com/tobymao/sqlglot/pull/4741) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#4740](https://github.com/tobymao/sqlglot/issues/4740) opened by [@martijnthe](https://github.com/martijnthe)*
- [`f0ce5ce`](https://github.com/tobymao/sqlglot/commit/f0ce5ce80d5a0ac24da4295a2f660bad9d7934be) - **snowflake**: unqualify ANY ORDER BY columns *(PR [#4737](https://github.com/tobymao/sqlglot/pull/4737) by [@georgesittas](https://github.com/georgesittas))*
- [`66b3ea9`](https://github.com/tobymao/sqlglot/commit/66b3ea905af34cfedc961c68ba738ba90d16221d) - **clickhouse**: respect type nullability when casting in strtodate_sql *(PR [#4744](https://github.com/tobymao/sqlglot/pull/4744) by [@sleshJdev](https://github.com/sleshJdev))*
- [`fda40a9`](https://github.com/tobymao/sqlglot/commit/fda40a9e670f30822dee05651d2336494c0d4f3b) - **clickhouse**: add BOTH if trim type is unspecified fixes [#4746](https://github.com/tobymao/sqlglot/pull/4746) *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`91f47fe`](https://github.com/tobymao/sqlglot/commit/91f47fec2a8c727f7e4c93fa54b6f06a36a6b42f) - Support for exp.HexString in DuckDB/Presto/Trino *(PR [#4743](https://github.com/tobymao/sqlglot/pull/4743) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4739](https://github.com/tobymao/sqlglot/issues/4739) opened by [@sk-](https://github.com/sk-)*
- [`f94656d`](https://github.com/tobymao/sqlglot/commit/f94656d06b118cee69ec8305fde948caa250cb69) - **clickhouse**: Generation of exp.ArrayConcat *(PR [#4754](https://github.com/tobymao/sqlglot/pull/4754) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4751](https://github.com/tobymao/sqlglot/issues/4751) opened by [@maximlipiev](https://github.com/maximlipiev)*
- [`cf33711`](https://github.com/tobymao/sqlglot/commit/cf337114a2259058b2a8cbde22d0a560001df9a5) - **tokenizer-rs**: use is_whitespace in token scan loop *(PR [#4756](https://github.com/tobymao/sqlglot/pull/4756) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#4748](https://github.com/tobymao/sqlglot/issues/4748) opened by [@gesoges0](https://github.com/gesoges0)*
- [`01008a9`](https://github.com/tobymao/sqlglot/commit/01008a91144a20e830df3783ff04beba7d029fec) - **mysql**: enable transpilation of CURDATE fixes [#4758](https://github.com/tobymao/sqlglot/pull/4758) *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`1fbd108`](https://github.com/tobymao/sqlglot/commit/1fbd108e8e9b85ae873bb702ff8f73c5ae94b392) - **sqlite**: correct json key value pair separator *(PR [#4774](https://github.com/tobymao/sqlglot/pull/4774) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *fixes issue [#4770](https://github.com/tobymao/sqlglot/issues/4770) opened by [@ori-n](https://github.com/ori-n)*
- [`ccdb92a`](https://github.com/tobymao/sqlglot/commit/ccdb92a7b676d3454cf9f6a18a2b0becec676169) - **optimizer**: do not qualify ctes that are pivoted *(PR [#4775](https://github.com/tobymao/sqlglot/pull/4775) by [@georgesittas](https://github.com/georgesittas))*
- [`ec4e97c`](https://github.com/tobymao/sqlglot/commit/ec4e97c31c16a1c8b5eb1acfba0ef77f064505b4) - **parser**: Do not preemptively create chunks for comment only statements *(PR [#4772](https://github.com/tobymao/sqlglot/pull/4772) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4762](https://github.com/tobymao/sqlglot/issues/4762) opened by [@dbittenbender](https://github.com/dbittenbender)*
- [`fc8aec3`](https://github.com/tobymao/sqlglot/commit/fc8aec35b211395ca603b2feabd1e5a5006fd677) - **parser**: Revert [#4772](https://github.com/tobymao/sqlglot/pull/4772) *(PR [#4783](https://github.com/tobymao/sqlglot/pull/4783) by [@VaggelisD](https://github.com/VaggelisD))*
- [`dc69192`](https://github.com/tobymao/sqlglot/commit/dc691923655d022f2a4b23e9df24a9d1518e7048) - **optimizer**: column qualify not applying in an order by inside within group when it also the alias *(PR [#4780](https://github.com/tobymao/sqlglot/pull/4780) by [@ran-lakeway](https://github.com/ran-lakeway))*
- [`ae00c92`](https://github.com/tobymao/sqlglot/commit/ae00c9203197a436bdb81289580eba0e7fc374ec) - **postgres**: properly alias generate series *(PR [#4789](https://github.com/tobymao/sqlglot/pull/4789) by [@georgesittas](https://github.com/georgesittas))*
- [`14c3de9`](https://github.com/tobymao/sqlglot/commit/14c3de91072435a68dd8d0f024dff0a0fd236cb8) - **optimizer**: parse unqualified names in DISTINCT ON (...) as identifiers *(PR [#4795](https://github.com/tobymao/sqlglot/pull/4795) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#4791](https://github.com/tobymao/sqlglot/issues/4791) opened by [@Fosly](https://github.com/Fosly)*
- [`00c7f05`](https://github.com/tobymao/sqlglot/commit/00c7f05f348eeb6b60d4e70d7e29e564caf38f64) - **optimizer**: do not coerce parameterized types *(PR [#4796](https://github.com/tobymao/sqlglot/pull/4796) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#4792](https://github.com/tobymao/sqlglot/issues/4792) opened by [@georgesittas](https://github.com/georgesittas)*
- [`9f6b4f7`](https://github.com/tobymao/sqlglot/commit/9f6b4f7dd6ea8eb80d8d5d3b75eeff9e262ceba2) - **snowflake**: add missing clauses in SHOW statements *(PR [#4797](https://github.com/tobymao/sqlglot/pull/4797) by [@georgesittas](https://github.com/georgesittas))*
- [`a9ae2d2`](https://github.com/tobymao/sqlglot/commit/a9ae2d229640f22134410ca5826c6f19df7ef523) - **tsql**: support TOP with PERCENT and WITH TIES *(PR [#4801](https://github.com/tobymao/sqlglot/pull/4801) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *fixes issue [#4793](https://github.com/tobymao/sqlglot/issues/4793) opened by [@gleb-daax](https://github.com/gleb-daax)*
- [`e05983a`](https://github.com/tobymao/sqlglot/commit/e05983a62ee9b3689e8a277924f2b1e36744a2a9) - **tokenizer-rs**: make hex literal tokenization more robust *(PR [#4802](https://github.com/tobymao/sqlglot/pull/4802) by [@georgesittas](https://github.com/georgesittas))*
### :wrench: Chores
- [`6c4c69e`](https://github.com/tobymao/sqlglot/commit/6c4c69e95d504ec839f7abd571eb2f614ea3c9ca) - move t-sql generators to base, clean up set_sql *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`4bf897d`](https://github.com/tobymao/sqlglot/commit/4bf897da6332175c22b772beb0b0cb6e6e941ebb) - remove redundant "parts" arg from CH ast, move gen methods to base *(PR [#4779](https://github.com/tobymao/sqlglot/pull/4779) by [@georgesittas](https://github.com/georgesittas))*
- [`e9c8eae`](https://github.com/tobymao/sqlglot/commit/e9c8eae060c05d02f6f65916d59f1792098679c8) - bump sqlglotrs to 0.4.0 *(commit by [@georgesittas](https://github.com/georgesittas))*
## [v26.6.0] - 2025-02-10
### :boom: BREAKING CHANGES
- due to [`a790e41`](https://github.com/tobymao/sqlglot/commit/a790e41736884bde7a8172f458db92d80064556f) - avoid redundant casts in FROM/TO_UTC_TIMESTAMP *(PR [#4725](https://github.com/tobymao/sqlglot/pull/4725) by [@georgesittas](https://github.com/georgesittas))*:
avoid redundant casts in FROM/TO_UTC_TIMESTAMP (#4725)
### :bug: Bug Fixes
- [`a790e41`](https://github.com/tobymao/sqlglot/commit/a790e41736884bde7a8172f458db92d80064556f) - **spark**: avoid redundant casts in FROM/TO_UTC_TIMESTAMP *(PR [#4725](https://github.com/tobymao/sqlglot/pull/4725) by [@georgesittas](https://github.com/georgesittas))*
## [v26.5.0] - 2025-02-10
### :boom: BREAKING CHANGES
- due to [`da52181`](https://github.com/tobymao/sqlglot/commit/da52181f1cd3ec22e5ac597de50036278d2e66e5) - TO_DATE parsing with safe flag true *(PR [#4713](https://github.com/tobymao/sqlglot/pull/4713) by [@geooo109](https://github.com/geooo109))*:
@ -5873,3 +6001,5 @@ Changelog
[v26.4.0]: https://github.com/tobymao/sqlglot/compare/v26.3.9...v26.4.0
[v26.4.1]: https://github.com/tobymao/sqlglot/compare/v26.4.0...v26.4.1
[v26.5.0]: https://github.com/tobymao/sqlglot/compare/v26.4.1...v26.5.0
[v26.6.0]: https://github.com/tobymao/sqlglot/compare/v26.5.0...v26.6.0
[v26.7.0]: https://github.com/tobymao/sqlglot/compare/v26.6.0...v26.7.0

2
docs/search.js
View file

File diff suppressed because one or more lines are too long

62
docs/sqlglot/_version.html
View file

@ -31,7 +31,10 @@
<h2>API Documentation</h2>
<ul class="memberlist">
<li>
<a class="variable" href="#TYPE_CHECKING">TYPE_CHECKING</a>
<a class="variable" href="#__version__">__version__</a>
</li>
<li>
<a class="variable" href="#__version_tuple__">__version_tuple__</a>
</li>
<li>
<a class="variable" href="#version">version</a>
@ -62,34 +65,51 @@
<label class="view-source-button" for="mod-_version-view-source"><span>View Source</span></label>
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="c1"># file generated by setuptools_scm</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="c1"># file generated by setuptools-scm</span>
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="c1"># don&#39;t change, don&#39;t track in version control</span>
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="n">TYPE_CHECKING</span> <span class="o">=</span> <span class="kc">False</span>
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="k">if</span> <span class="n">TYPE_CHECKING</span><span class="p">:</span>
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a> <span class="kn">from</span><span class="w"> </span><span class="nn">typing</span><span class="w"> </span><span class="kn">import</span> <span class="n">Tuple</span><span class="p">,</span> <span class="n">Union</span>
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a> <span class="n">VERSION_TUPLE</span> <span class="o">=</span> <span class="n">Tuple</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="nb">str</span><span class="p">],</span> <span class="o">...</span><span class="p">]</span>
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="k">else</span><span class="p">:</span>
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a> <span class="n">VERSION_TUPLE</span> <span class="o">=</span> <span class="nb">object</span>
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a>
</span><span id="L-10"><a href="#L-10"><span class="linenos">10</span></a><span class="n">version</span><span class="p">:</span> <span class="nb">str</span>
</span><span id="L-11"><a href="#L-11"><span class="linenos">11</span></a><span class="n">__version__</span><span class="p">:</span> <span class="nb">str</span>
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a>
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="n">__all__</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;__version__&quot;</span><span class="p">,</span> <span class="s2">&quot;__version_tuple__&quot;</span><span class="p">,</span> <span class="s2">&quot;version&quot;</span><span class="p">,</span> <span class="s2">&quot;version_tuple&quot;</span><span class="p">]</span>
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a>
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="n">TYPE_CHECKING</span> <span class="o">=</span> <span class="kc">False</span>
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="k">if</span> <span class="n">TYPE_CHECKING</span><span class="p">:</span>
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a> <span class="kn">from</span><span class="w"> </span><span class="nn">typing</span><span class="w"> </span><span class="kn">import</span> <span class="n">Tuple</span>
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a> <span class="kn">from</span><span class="w"> </span><span class="nn">typing</span><span class="w"> </span><span class="kn">import</span> <span class="n">Union</span>
</span><span id="L-10"><a href="#L-10"><span class="linenos">10</span></a>
</span><span id="L-11"><a href="#L-11"><span class="linenos">11</span></a> <span class="n">VERSION_TUPLE</span> <span class="o">=</span> <span class="n">Tuple</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="nb">str</span><span class="p">],</span> <span class="o">...</span><span class="p">]</span>
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="k">else</span><span class="p">:</span>
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a> <span class="n">VERSION_TUPLE</span> <span class="o">=</span> <span class="nb">object</span>
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a>
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;26.5.0&#39;</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">26</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">version</span><span class="p">:</span> <span class="nb">str</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version__</span><span class="p">:</span> <span class="nb">str</span>
</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a>
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;26.7.0&#39;</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">26</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
</span></pre></div>
</section>
<section id="TYPE_CHECKING">
<section id="__version__">
<div class="attr variable">
<span class="name">TYPE_CHECKING</span> =
<span class="default_value">False</span>
<span class="name">__version__</span><span class="annotation">: str</span> =
<span class="default_value">&#39;26.7.0&#39;</span>
</div>
<a class="headerlink" href="#TYPE_CHECKING"></a>
<a class="headerlink" href="#__version__"></a>
</section>
<section id="__version_tuple__">
<div class="attr variable">
<span class="name">__version_tuple__</span><span class="annotation">: object</span> =
<span class="default_value">(26, 7, 0)</span>
</div>
<a class="headerlink" href="#__version_tuple__"></a>
@ -97,7 +117,7 @@
<section id="version">
<div class="attr variable">
<span class="name">version</span><span class="annotation">: str</span> =
<span class="default_value">&#39;26.5.0&#39;</span>
<span class="default_value">&#39;26.7.0&#39;</span>
</div>
@ -109,7 +129,7 @@
<section id="version_tuple">
<div class="attr variable">
<span class="name">version_tuple</span><span class="annotation">: object</span> =
<span class="default_value">(26, 5, 0)</span>
<span class="default_value">(26, 7, 0)</span>
</div>

58
docs/sqlglot/dialects.html
View file

@ -325,7 +325,7 @@ dialect implementations in order to understand how their various components can
<section id="Athena">
<div class="attr variable">
<span class="name">Athena</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990150988080&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462755349024&#39;&gt;</span>
</div>
@ -337,7 +337,7 @@ dialect implementations in order to understand how their various components can
<section id="BigQuery">
<div class="attr variable">
<span class="name">BigQuery</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990155292544&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462741502272&#39;&gt;</span>
</div>
@ -349,7 +349,7 @@ dialect implementations in order to understand how their various components can
<section id="ClickHouse">
<div class="attr variable">
<span class="name">ClickHouse</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990146633968&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462754762032&#39;&gt;</span>
</div>
@ -361,7 +361,7 @@ dialect implementations in order to understand how their various components can
<section id="Databricks">
<div class="attr variable">
<span class="name">Databricks</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990146641264&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462742589632&#39;&gt;</span>
</div>
@ -373,7 +373,7 @@ dialect implementations in order to understand how their various components can
<section id="Doris">
<div class="attr variable">
<span class="name">Doris</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990155444768&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462742590208&#39;&gt;</span>
</div>
@ -385,7 +385,7 @@ dialect implementations in order to understand how their various components can
<section id="Drill">
<div class="attr variable">
<span class="name">Drill</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990142497552&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462740790416&#39;&gt;</span>
</div>
@ -397,7 +397,7 @@ dialect implementations in order to understand how their various components can
<section id="Druid">
<div class="attr variable">
<span class="name">Druid</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990140097664&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462740798336&#39;&gt;</span>
</div>
@ -409,7 +409,7 @@ dialect implementations in order to understand how their various components can
<section id="DuckDB">
<div class="attr variable">
<span class="name">DuckDB</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990145643760&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462741187664&#39;&gt;</span>
</div>
@ -421,7 +421,7 @@ dialect implementations in order to understand how their various components can
<section id="Dune">
<div class="attr variable">
<span class="name">Dune</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990155886064&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462745526160&#39;&gt;</span>
</div>
@ -433,7 +433,7 @@ dialect implementations in order to understand how their various components can
<section id="Hive">
<div class="attr variable">
<span class="name">Hive</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990155879632&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462750759136&#39;&gt;</span>
</div>
@ -445,7 +445,7 @@ dialect implementations in order to understand how their various components can
<section id="Materialize">
<div class="attr variable">
<span class="name">Materialize</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990155535520&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462751357952&#39;&gt;</span>
</div>
@ -457,7 +457,7 @@ dialect implementations in order to understand how their various components can
<section id="MySQL">
<div class="attr variable">
<span class="name">MySQL</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990145679760&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462751363424&#39;&gt;</span>
</div>
@ -469,7 +469,7 @@ dialect implementations in order to understand how their various components can
<section id="Oracle">
<div class="attr variable">
<span class="name">Oracle</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990144694544&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462755217712&#39;&gt;</span>
</div>
@ -481,7 +481,7 @@ dialect implementations in order to understand how their various components can
<section id="Postgres">
<div class="attr variable">
<span class="name">Postgres</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990144681200&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462755233024&#39;&gt;</span>
</div>
@ -493,7 +493,7 @@ dialect implementations in order to understand how their various components can
<section id="Presto">
<div class="attr variable">
<span class="name">Presto</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990155130576&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462756443568&#39;&gt;</span>
</div>
@ -505,7 +505,7 @@ dialect implementations in order to understand how their various components can
<section id="PRQL">
<div class="attr variable">
<span class="name">PRQL</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990156080368&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462755122432&#39;&gt;</span>
</div>
@ -517,7 +517,7 @@ dialect implementations in order to understand how their various components can
<section id="Redshift">
<div class="attr variable">
<span class="name">Redshift</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990156070864&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462755125120&#39;&gt;</span>
</div>
@ -529,7 +529,7 @@ dialect implementations in order to understand how their various components can
<section id="RisingWave">
<div class="attr variable">
<span class="name">RisingWave</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990147260640&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462756817904&#39;&gt;</span>
</div>
@ -541,7 +541,7 @@ dialect implementations in order to understand how their various components can
<section id="Snowflake">
<div class="attr variable">
<span class="name">Snowflake</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990147258336&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462743195648&#39;&gt;</span>
</div>
@ -553,7 +553,7 @@ dialect implementations in order to understand how their various components can
<section id="Spark">
<div class="attr variable">
<span class="name">Spark</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990155948672&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462755039024&#39;&gt;</span>
</div>
@ -565,7 +565,7 @@ dialect implementations in order to understand how their various components can
<section id="Spark2">
<div class="attr variable">
<span class="name">Spark2</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990154264192&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462746722960&#39;&gt;</span>
</div>
@ -577,7 +577,7 @@ dialect implementations in order to understand how their various components can
<section id="SQLite">
<div class="attr variable">
<span class="name">SQLite</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990145087616&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462742002304&#39;&gt;</span>
</div>
@ -589,7 +589,7 @@ dialect implementations in order to understand how their various components can
<section id="StarRocks">
<div class="attr variable">
<span class="name">StarRocks</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990142730592&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462742011664&#39;&gt;</span>
</div>
@ -601,7 +601,7 @@ dialect implementations in order to understand how their various components can
<section id="Tableau">
<div class="attr variable">
<span class="name">Tableau</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990155788928&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462741692736&#39;&gt;</span>
</div>
@ -613,7 +613,7 @@ dialect implementations in order to understand how their various components can
<section id="Teradata">
<div class="attr variable">
<span class="name">Teradata</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990146321568&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462745788496&#39;&gt;</span>
</div>
@ -625,7 +625,7 @@ dialect implementations in order to understand how their various components can
<section id="Trino">
<div class="attr variable">
<span class="name">Trino</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990140471472&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462753877488&#39;&gt;</span>
</div>
@ -637,7 +637,7 @@ dialect implementations in order to understand how their various components can
<section id="TSQL">
<div class="attr variable">
<span class="name">TSQL</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990140479392&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462753885648&#39;&gt;</span>
</div>
@ -649,7 +649,7 @@ dialect implementations in order to understand how their various components can
<section id="Dialect">
<div class="attr variable">
<span class="name">Dialect</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990140487376&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462739461584&#39;&gt;</span>
</div>
@ -661,7 +661,7 @@ dialect implementations in order to understand how their various components can
<section id="Dialects">
<div class="attr variable">
<span class="name">Dialects</span> =
<span class="default_value">&lt;MagicMock id=&#39;139990140495296&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;140462739469504&#39;&gt;</span>
</div>

13
docs/sqlglot/executor/python.html
View file

File diff suppressed because one or more lines are too long

26980
docs/sqlglot/expressions.html
View file

File diff suppressed because one or more lines are too long

20195
docs/sqlglot/generator.html
View file

File diff suppressed because it is too large Load diff

2
docs/sqlglot/helper.html
View file

@ -1920,7 +1920,7 @@ belong to some totally-ordered set.</p>
<section id="DATE_UNITS">
<div class="attr variable">
<span class="name">DATE_UNITS</span> =
<span class="default_value">{&#39;month&#39;, &#39;day&#39;, &#39;week&#39;, &#39;quarter&#39;, &#39;year&#39;, &#39;year_month&#39;}</span>
<span class="default_value">{&#39;day&#39;, &#39;year_month&#39;, &#39;week&#39;, &#39;quarter&#39;, &#39;year&#39;, &#39;month&#39;}</span>
</div>

2
docs/sqlglot/jsonpath.html
View file

@ -641,7 +641,7 @@
<div class="attr variable">
<span class="name">ALL_JSON_PATH_PARTS</span> =
<input id="ALL_JSON_PATH_PARTS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>&#39;&gt;}</span>
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>&#39;&gt;}</span>
</div>

1030
docs/sqlglot/optimizer/annotate_types.html
View file

File diff suppressed because one or more lines are too long

2
docs/sqlglot/optimizer/merge_subqueries.html
View file

@ -581,7 +581,7 @@ queries if it would result in multiple table selects in a single query:</p>
<div class="attr variable">
<span class="name">UNMERGABLE_ARGS</span> =
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;qualify&#39;, &#39;sample&#39;, &#39;connect&#39;, &#39;limit&#39;, &#39;match&#39;, &#39;prewhere&#39;, &#39;locks&#39;, &#39;pivots&#39;, &#39;group&#39;, &#39;laterals&#39;, &#39;windows&#39;, &#39;operation_modifiers&#39;, &#39;having&#39;, &#39;format&#39;, &#39;settings&#39;, &#39;options&#39;, &#39;cluster&#39;, &#39;kind&#39;, &#39;offset&#39;, &#39;distribute&#39;, &#39;sort&#39;, &#39;distinct&#39;, &#39;with&#39;, &#39;into&#39;}</span>
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;sample&#39;, &#39;match&#39;, &#39;sort&#39;, &#39;locks&#39;, &#39;format&#39;, &#39;into&#39;, &#39;limit&#39;, &#39;offset&#39;, &#39;operation_modifiers&#39;, &#39;distinct&#39;, &#39;windows&#39;, &#39;having&#39;, &#39;laterals&#39;, &#39;settings&#39;, &#39;kind&#39;, &#39;distribute&#39;, &#39;connect&#39;, &#39;pivots&#39;, &#39;group&#39;, &#39;cluster&#39;, &#39;prewhere&#39;, &#39;options&#39;, &#39;qualify&#39;, &#39;with&#39;}</span>
</div>

196
docs/sqlglot/optimizer/qualify_tables.html
View file

@ -155,54 +155,58 @@
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">source</span><span class="o">.</span><span class="n">alias</span>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="p">)</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="p">)</span>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">if</span> <span class="n">infer_csv_schemas</span> <span class="ow">and</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="n">source</span><span class="p">,</span>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="p">)</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="p">)</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">))</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="p">):</span>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">if</span> <span class="n">pivots</span><span class="p">:</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="n">pivot_alias</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">pivot_alias</span><span class="p">)))</span>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="c1"># This case corresponds to a pivoted CTE, we don&#39;t want to qualify that</span>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">),</span> <span class="n">Scope</span><span class="p">):</span>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">continue</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">if</span> <span class="n">infer_csv_schemas</span> <span class="ow">and</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">source</span><span class="p">,</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="p">)</span>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="p">)</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">))</span>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="p">):</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;table&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;table&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a>
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>
@ -302,54 +306,58 @@
</span><span id="qualify_tables-98"><a href="#qualify_tables-98"><span class="linenos"> 98</span></a> <span class="n">source</span><span class="o">.</span><span class="n">alias</span>
</span><span id="qualify_tables-99"><a href="#qualify_tables-99"><span class="linenos"> 99</span></a> <span class="p">)</span>
</span><span id="qualify_tables-100"><a href="#qualify_tables-100"><span class="linenos">100</span></a>
</span><span id="qualify_tables-101"><a href="#qualify_tables-101"><span class="linenos">101</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
</span><span id="qualify_tables-102"><a href="#qualify_tables-102"><span class="linenos">102</span></a>
</span><span id="qualify_tables-103"><a href="#qualify_tables-103"><span class="linenos">103</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="qualify_tables-104"><a href="#qualify_tables-104"><span class="linenos">104</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
</span><span id="qualify_tables-105"><a href="#qualify_tables-105"><span class="linenos">105</span></a> <span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="qualify_tables-106"><a href="#qualify_tables-106"><span class="linenos">106</span></a> <span class="p">)</span>
</span><span id="qualify_tables-107"><a href="#qualify_tables-107"><span class="linenos">107</span></a>
</span><span id="qualify_tables-108"><a href="#qualify_tables-108"><span class="linenos">108</span></a> <span class="k">if</span> <span class="n">infer_csv_schemas</span> <span class="ow">and</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
</span><span id="qualify_tables-109"><a href="#qualify_tables-109"><span class="linenos">109</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
</span><span id="qualify_tables-110"><a href="#qualify_tables-110"><span class="linenos">110</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="qualify_tables-111"><a href="#qualify_tables-111"><span class="linenos">111</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="qualify_tables-112"><a href="#qualify_tables-112"><span class="linenos">112</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
</span><span id="qualify_tables-113"><a href="#qualify_tables-113"><span class="linenos">113</span></a> <span class="n">source</span><span class="p">,</span>
</span><span id="qualify_tables-114"><a href="#qualify_tables-114"><span class="linenos">114</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
</span><span id="qualify_tables-115"><a href="#qualify_tables-115"><span class="linenos">115</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="qualify_tables-116"><a href="#qualify_tables-116"><span class="linenos">116</span></a> <span class="p">)</span>
</span><span id="qualify_tables-117"><a href="#qualify_tables-117"><span class="linenos">117</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
</span><span id="qualify_tables-118"><a href="#qualify_tables-118"><span class="linenos">118</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="qualify_tables-119"><a href="#qualify_tables-119"><span class="linenos">119</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="qualify_tables-120"><a href="#qualify_tables-120"><span class="linenos">120</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
</span><span id="qualify_tables-121"><a href="#qualify_tables-121"><span class="linenos">121</span></a> <span class="p">)</span>
</span><span id="qualify_tables-122"><a href="#qualify_tables-122"><span class="linenos">122</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="qualify_tables-123"><a href="#qualify_tables-123"><span class="linenos">123</span></a>
</span><span id="qualify_tables-124"><a href="#qualify_tables-124"><span class="linenos">124</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
</span><span id="qualify_tables-125"><a href="#qualify_tables-125"><span class="linenos">125</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="qualify_tables-126"><a href="#qualify_tables-126"><span class="linenos">126</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="qualify_tables-127"><a href="#qualify_tables-127"><span class="linenos">127</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
</span><span id="qualify_tables-128"><a href="#qualify_tables-128"><span class="linenos">128</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">))</span>
</span><span id="qualify_tables-129"><a href="#qualify_tables-129"><span class="linenos">129</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="qualify_tables-130"><a href="#qualify_tables-130"><span class="linenos">130</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
</span><span id="qualify_tables-131"><a href="#qualify_tables-131"><span class="linenos">131</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="qualify_tables-132"><a href="#qualify_tables-132"><span class="linenos">132</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
</span><span id="qualify_tables-133"><a href="#qualify_tables-133"><span class="linenos">133</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span>
</span><span id="qualify_tables-134"><a href="#qualify_tables-134"><span class="linenos">134</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
</span><span id="qualify_tables-135"><a href="#qualify_tables-135"><span class="linenos">135</span></a> <span class="p">):</span>
</span><span id="qualify_tables-136"><a href="#qualify_tables-136"><span class="linenos">136</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span>
</span><span id="qualify_tables-137"><a href="#qualify_tables-137"><span class="linenos">137</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="qualify_tables-138"><a href="#qualify_tables-138"><span class="linenos">138</span></a>
</span><span id="qualify_tables-139"><a href="#qualify_tables-139"><span class="linenos">139</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="qualify_tables-140"><a href="#qualify_tables-140"><span class="linenos">140</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span>
</span><span id="qualify_tables-141"><a href="#qualify_tables-141"><span class="linenos">141</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
</span><span id="qualify_tables-101"><a href="#qualify_tables-101"><span class="linenos">101</span></a> <span class="k">if</span> <span class="n">pivots</span><span class="p">:</span>
</span><span id="qualify_tables-102"><a href="#qualify_tables-102"><span class="linenos">102</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="qualify_tables-103"><a href="#qualify_tables-103"><span class="linenos">103</span></a> <span class="n">pivot_alias</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
</span><span id="qualify_tables-104"><a href="#qualify_tables-104"><span class="linenos">104</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">pivot_alias</span><span class="p">)))</span>
</span><span id="qualify_tables-105"><a href="#qualify_tables-105"><span class="linenos">105</span></a>
</span><span id="qualify_tables-106"><a href="#qualify_tables-106"><span class="linenos">106</span></a> <span class="c1"># This case corresponds to a pivoted CTE, we don&#39;t want to qualify that</span>
</span><span id="qualify_tables-107"><a href="#qualify_tables-107"><span class="linenos">107</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">),</span> <span class="n">Scope</span><span class="p">):</span>
</span><span id="qualify_tables-108"><a href="#qualify_tables-108"><span class="linenos">108</span></a> <span class="k">continue</span>
</span><span id="qualify_tables-109"><a href="#qualify_tables-109"><span class="linenos">109</span></a>
</span><span id="qualify_tables-110"><a href="#qualify_tables-110"><span class="linenos">110</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
</span><span id="qualify_tables-111"><a href="#qualify_tables-111"><span class="linenos">111</span></a>
</span><span id="qualify_tables-112"><a href="#qualify_tables-112"><span class="linenos">112</span></a> <span class="k">if</span> <span class="n">infer_csv_schemas</span> <span class="ow">and</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
</span><span id="qualify_tables-113"><a href="#qualify_tables-113"><span class="linenos">113</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
</span><span id="qualify_tables-114"><a href="#qualify_tables-114"><span class="linenos">114</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="qualify_tables-115"><a href="#qualify_tables-115"><span class="linenos">115</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="qualify_tables-116"><a href="#qualify_tables-116"><span class="linenos">116</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
</span><span id="qualify_tables-117"><a href="#qualify_tables-117"><span class="linenos">117</span></a> <span class="n">source</span><span class="p">,</span>
</span><span id="qualify_tables-118"><a href="#qualify_tables-118"><span class="linenos">118</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
</span><span id="qualify_tables-119"><a href="#qualify_tables-119"><span class="linenos">119</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="qualify_tables-120"><a href="#qualify_tables-120"><span class="linenos">120</span></a> <span class="p">)</span>
</span><span id="qualify_tables-121"><a href="#qualify_tables-121"><span class="linenos">121</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
</span><span id="qualify_tables-122"><a href="#qualify_tables-122"><span class="linenos">122</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="qualify_tables-123"><a href="#qualify_tables-123"><span class="linenos">123</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="qualify_tables-124"><a href="#qualify_tables-124"><span class="linenos">124</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
</span><span id="qualify_tables-125"><a href="#qualify_tables-125"><span class="linenos">125</span></a> <span class="p">)</span>
</span><span id="qualify_tables-126"><a href="#qualify_tables-126"><span class="linenos">126</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="qualify_tables-127"><a href="#qualify_tables-127"><span class="linenos">127</span></a>
</span><span id="qualify_tables-128"><a href="#qualify_tables-128"><span class="linenos">128</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
</span><span id="qualify_tables-129"><a href="#qualify_tables-129"><span class="linenos">129</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="qualify_tables-130"><a href="#qualify_tables-130"><span class="linenos">130</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="qualify_tables-131"><a href="#qualify_tables-131"><span class="linenos">131</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
</span><span id="qualify_tables-132"><a href="#qualify_tables-132"><span class="linenos">132</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">))</span>
</span><span id="qualify_tables-133"><a href="#qualify_tables-133"><span class="linenos">133</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="qualify_tables-134"><a href="#qualify_tables-134"><span class="linenos">134</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
</span><span id="qualify_tables-135"><a href="#qualify_tables-135"><span class="linenos">135</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="qualify_tables-136"><a href="#qualify_tables-136"><span class="linenos">136</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
</span><span id="qualify_tables-137"><a href="#qualify_tables-137"><span class="linenos">137</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span>
</span><span id="qualify_tables-138"><a href="#qualify_tables-138"><span class="linenos">138</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
</span><span id="qualify_tables-139"><a href="#qualify_tables-139"><span class="linenos">139</span></a> <span class="p">):</span>
</span><span id="qualify_tables-140"><a href="#qualify_tables-140"><span class="linenos">140</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span>
</span><span id="qualify_tables-141"><a href="#qualify_tables-141"><span class="linenos">141</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="qualify_tables-142"><a href="#qualify_tables-142"><span class="linenos">142</span></a>
</span><span id="qualify_tables-143"><a href="#qualify_tables-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span>
</span><span id="qualify_tables-144"><a href="#qualify_tables-144"><span class="linenos">144</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
</span><span id="qualify_tables-145"><a href="#qualify_tables-145"><span class="linenos">145</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="qualify_tables-146"><a href="#qualify_tables-146"><span class="linenos">146</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;table&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="qualify_tables-147"><a href="#qualify_tables-147"><span class="linenos">147</span></a>
</span><span id="qualify_tables-148"><a href="#qualify_tables-148"><span class="linenos">148</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="qualify_tables-143"><a href="#qualify_tables-143"><span class="linenos">143</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="qualify_tables-144"><a href="#qualify_tables-144"><span class="linenos">144</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span>
</span><span id="qualify_tables-145"><a href="#qualify_tables-145"><span class="linenos">145</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
</span><span id="qualify_tables-146"><a href="#qualify_tables-146"><span class="linenos">146</span></a>
</span><span id="qualify_tables-147"><a href="#qualify_tables-147"><span class="linenos">147</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span>
</span><span id="qualify_tables-148"><a href="#qualify_tables-148"><span class="linenos">148</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
</span><span id="qualify_tables-149"><a href="#qualify_tables-149"><span class="linenos">149</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="qualify_tables-150"><a href="#qualify_tables-150"><span class="linenos">150</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;table&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="qualify_tables-151"><a href="#qualify_tables-151"><span class="linenos">151</span></a>
</span><span id="qualify_tables-152"><a href="#qualify_tables-152"><span class="linenos">152</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>

6
docs/sqlglot/optimizer/scope.html
View file

@ -542,7 +542,7 @@
</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a> <span class="ow">or</span> <span class="p">(</span>
</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ancestor</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Order</span><span class="p">)</span>
</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a> <span class="ow">and</span> <span class="p">(</span>
</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ancestor</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ancestor</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">WithinGroup</span><span class="p">))</span>
</span><span id="L-296"><a href="#L-296"><span class="linenos">296</span></a> <span class="ow">or</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">named_selects</span>
</span><span id="L-297"><a href="#L-297"><span class="linenos">297</span></a> <span class="p">)</span>
</span><span id="L-298"><a href="#L-298"><span class="linenos">298</span></a> <span class="p">)</span>
@ -1548,7 +1548,7 @@
</span><span id="Scope-293"><a href="#Scope-293"><span class="linenos">293</span></a> <span class="ow">or</span> <span class="p">(</span>
</span><span id="Scope-294"><a href="#Scope-294"><span class="linenos">294</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ancestor</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Order</span><span class="p">)</span>
</span><span id="Scope-295"><a href="#Scope-295"><span class="linenos">295</span></a> <span class="ow">and</span> <span class="p">(</span>
</span><span id="Scope-296"><a href="#Scope-296"><span class="linenos">296</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ancestor</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
</span><span id="Scope-296"><a href="#Scope-296"><span class="linenos">296</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ancestor</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">WithinGroup</span><span class="p">))</span>
</span><span id="Scope-297"><a href="#Scope-297"><span class="linenos">297</span></a> <span class="ow">or</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">named_selects</span>
</span><span id="Scope-298"><a href="#Scope-298"><span class="linenos">298</span></a> <span class="p">)</span>
</span><span id="Scope-299"><a href="#Scope-299"><span class="linenos">299</span></a> <span class="p">)</span>
@ -2415,7 +2415,7 @@ a list of the left and right child scopes.</li>
</span><span id="Scope.columns-293"><a href="#Scope.columns-293"><span class="linenos">293</span></a> <span class="ow">or</span> <span class="p">(</span>
</span><span id="Scope.columns-294"><a href="#Scope.columns-294"><span class="linenos">294</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ancestor</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Order</span><span class="p">)</span>
</span><span id="Scope.columns-295"><a href="#Scope.columns-295"><span class="linenos">295</span></a> <span class="ow">and</span> <span class="p">(</span>
</span><span id="Scope.columns-296"><a href="#Scope.columns-296"><span class="linenos">296</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ancestor</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
</span><span id="Scope.columns-296"><a href="#Scope.columns-296"><span class="linenos">296</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ancestor</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">WithinGroup</span><span class="p">))</span>
</span><span id="Scope.columns-297"><a href="#Scope.columns-297"><span class="linenos">297</span></a> <span class="ow">or</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">named_selects</span>
</span><span id="Scope.columns-298"><a href="#Scope.columns-298"><span class="linenos">298</span></a> <span class="p">)</span>
</span><span id="Scope.columns-299"><a href="#Scope.columns-299"><span class="linenos">299</span></a> <span class="p">)</span>

4
docs/sqlglot/optimizer/simplify.html
View file

@ -3238,7 +3238,7 @@ prefix are statically known.</p>
<div class="attr variable">
<span class="name">DATETRUNC_COMPARISONS</span> =
<input id="DATETRUNC_COMPARISONS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#In">sqlglot.expressions.In</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;}</span>
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="../expressions.html#In">sqlglot.expressions.In</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;}</span>
</div>
@ -3322,7 +3322,7 @@ prefix are statically known.</p>
<section id="JOINS">
<div class="attr variable">
<span class="name">JOINS</span> =
<span class="default_value">{(&#39;&#39;, &#39;INNER&#39;), (&#39;RIGHT&#39;, &#39;&#39;), (&#39;RIGHT&#39;, &#39;OUTER&#39;), (&#39;&#39;, &#39;&#39;)}</span>
<span class="default_value">{(&#39;RIGHT&#39;, &#39;&#39;), (&#39;&#39;, &#39;INNER&#39;), (&#39;RIGHT&#39;, &#39;OUTER&#39;), (&#39;&#39;, &#39;&#39;)}</span>
</div>

30109
docs/sqlglot/parser.html
View file

File diff suppressed because one or more lines are too long

2
docs/sqlglot/time.html
View file

File diff suppressed because one or more lines are too long

5784
docs/sqlglot/tokens.html
View file

File diff suppressed because it is too large Load diff

4
docs/sqlglot/transforms.html
View file

@ -391,7 +391,7 @@
</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a> <span class="p">}</span>
</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a>
</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a> <span class="n">select_candidates</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span> <span class="k">if</span> <span class="n">expression</span><span class="o">.</span><span class="n">is_star</span> <span class="k">else</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span>
</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a> <span class="k">for</span> <span class="n">select_candidate</span> <span class="ow">in</span> <span class="n">qualify_filters</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">select_candidates</span><span class="p">):</span>
</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a> <span class="k">for</span> <span class="n">select_candidate</span> <span class="ow">in</span> <span class="nb">list</span><span class="p">(</span><span class="n">qualify_filters</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">select_candidates</span><span class="p">)):</span>
</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select_candidate</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">):</span>
</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">expression_by_alias</span><span class="p">:</span>
</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">select_candidate</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">):</span>
@ -1496,7 +1496,7 @@ or the appropriate <code>Generator.TRANSFORMS</code> function (when applicable -
</span><span id="eliminate_qualify-259"><a href="#eliminate_qualify-259"><span class="linenos">259</span></a> <span class="p">}</span>
</span><span id="eliminate_qualify-260"><a href="#eliminate_qualify-260"><span class="linenos">260</span></a>
</span><span id="eliminate_qualify-261"><a href="#eliminate_qualify-261"><span class="linenos">261</span></a> <span class="n">select_candidates</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span> <span class="k">if</span> <span class="n">expression</span><span class="o">.</span><span class="n">is_star</span> <span class="k">else</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span>
</span><span id="eliminate_qualify-262"><a href="#eliminate_qualify-262"><span class="linenos">262</span></a> <span class="k">for</span> <span class="n">select_candidate</span> <span class="ow">in</span> <span class="n">qualify_filters</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">select_candidates</span><span class="p">):</span>
</span><span id="eliminate_qualify-262"><a href="#eliminate_qualify-262"><span class="linenos">262</span></a> <span class="k">for</span> <span class="n">select_candidate</span> <span class="ow">in</span> <span class="nb">list</span><span class="p">(</span><span class="n">qualify_filters</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">select_candidates</span><span class="p">)):</span>
</span><span id="eliminate_qualify-263"><a href="#eliminate_qualify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select_candidate</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">):</span>
</span><span id="eliminate_qualify-264"><a href="#eliminate_qualify-264"><span class="linenos">264</span></a> <span class="k">if</span> <span class="n">expression_by_alias</span><span class="p">:</span>
</span><span id="eliminate_qualify-265"><a href="#eliminate_qualify-265"><span class="linenos">265</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">select_candidate</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">):</span>

2
sqlglot/dialects/bigquery.py
View file

@ -356,6 +356,7 @@ class BigQuery(Dialect):
FORCE_EARLY_ALIAS_REF_EXPANSION = True
EXPAND_ALIAS_REFS_EARLY_ONLY_IN_GROUP_BY = True
PRESERVE_ORIGINAL_NAMES = True
HEX_STRING_IS_INTEGER_TYPE = True
# https://cloud.google.com/bigquery/docs/reference/standard-sql/lexical#case_sensitivity
NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
@ -903,6 +904,7 @@ class BigQuery(Dialect):
exp.GenerateSeries: rename_func("GENERATE_ARRAY"),
exp.GroupConcat: rename_func("STRING_AGG"),
exp.Hex: lambda self, e: self.func("UPPER", self.func("TO_HEX", self.sql(e, "this"))),
exp.HexString: lambda self, e: self.hexstring_sql(e, binary_function_repr="FROM_HEX"),
exp.If: if_sql(false_value="NULL"),
exp.ILike: no_ilike_sql,
exp.IntDiv: rename_func("DIV"),

57
sqlglot/dialects/clickhouse.py
View file

@ -82,7 +82,7 @@ def _build_count_if(args: t.List) -> exp.CountIf | exp.CombinedAggFunc:
if len(args) == 1:
return exp.CountIf(this=seq_get(args, 0))
return exp.CombinedAggFunc(this="countIf", expressions=args, parts=("count", "If"))
return exp.CombinedAggFunc(this="countIf", expressions=args)
def _build_str_to_date(args: t.List) -> exp.Cast | exp.Anonymous:
@ -190,6 +190,7 @@ class ClickHouse(Dialect):
PRESERVE_ORIGINAL_NAMES = True
NUMBERS_CAN_BE_UNDERSCORE_SEPARATED = True
IDENTIFIERS_CAN_START_WITH_DIGIT = True
HEX_STRING_IS_INTEGER_TYPE = True
# https://github.com/ClickHouse/ClickHouse/issues/33935#issue-1112165779
NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_SENSITIVE
@ -217,6 +218,7 @@ class ClickHouse(Dialect):
KEYWORDS = {
**tokens.Tokenizer.KEYWORDS,
".:": TokenType.DOTCOLON,
"ATTACH": TokenType.COMMAND,
"DATE32": TokenType.DATE32,
"DATETIME64": TokenType.DATETIME64,
@ -438,6 +440,8 @@ class ClickHouse(Dialect):
FUNC_TOKENS = {
*parser.Parser.FUNC_TOKENS,
TokenType.AND,
TokenType.OR,
TokenType.SET,
}
@ -477,8 +481,7 @@ class ClickHouse(Dialect):
RANGE_PARSERS = {
**parser.Parser.RANGE_PARSERS,
TokenType.GLOBAL: lambda self, this: self._match(TokenType.IN)
and self._parse_in(this, is_global=True),
TokenType.GLOBAL: lambda self, this: self._parse_global_in(this),
}
# The PLACEHOLDER entry is popped because 1) it doesn't affect Clickhouse (it corresponds to
@ -632,6 +635,11 @@ class ClickHouse(Dialect):
this.set("is_global", is_global)
return this
def _parse_global_in(self, this: t.Optional[exp.Expression]) -> exp.Not | exp.In:
is_negated = self._match(TokenType.NOT)
this = self._match(TokenType.IN) and self._parse_in(this, is_global=True)
return self.expression(exp.Not, this=this) if is_negated else this
def _parse_table(
self,
schema: bool = False,
@ -649,6 +657,13 @@ class ClickHouse(Dialect):
is_db_reference=is_db_reference,
)
if isinstance(this, exp.Table):
inner = this.this
alias = this.args.get("alias")
if isinstance(inner, exp.GenerateSeries) and alias and not alias.columns:
alias.set("columns", [exp.to_identifier("generate_series")])
if self._match(TokenType.FINAL):
this = self.expression(exp.Final, this=this)
@ -735,7 +750,6 @@ class ClickHouse(Dialect):
"expressions": anon_func.expressions,
}
if parts[1]:
kwargs["parts"] = parts
exp_class: t.Type[exp.Expression] = (
exp.CombinedParameterizedAgg if params else exp.CombinedAggFunc
)
@ -902,7 +916,6 @@ class ClickHouse(Dialect):
TABLE_HINTS = False
GROUPINGS_SEP = ""
SET_OP_MODIFIERS = False
SUPPORTS_TABLE_ALIAS_COLUMNS = False
VALUES_AS_TABLE = False
ARRAY_SIZE_NAME = "LENGTH"
@ -982,6 +995,7 @@ class ClickHouse(Dialect):
**generator.Generator.TRANSFORMS,
exp.AnyValue: rename_func("any"),
exp.ApproxDistinct: rename_func("uniq"),
exp.ArrayConcat: rename_func("arrayConcat"),
exp.ArrayFilter: lambda self, e: self.func("arrayFilter", e.expression, e.this),
exp.ArraySum: rename_func("arraySum"),
exp.ArgMax: arg_max_or_min_no_count("argMax"),
@ -1001,6 +1015,7 @@ class ClickHouse(Dialect):
exp.Explode: rename_func("arrayJoin"),
exp.Final: lambda self, e: f"{self.sql(e, 'this')} FINAL",
exp.IsNan: rename_func("isNaN"),
exp.JSONCast: lambda self, e: f"{self.sql(e, 'this')}.:{self.sql(e, 'to')}",
exp.JSONExtract: json_extract_segments("JSONExtractString", quoted_index=False),
exp.JSONExtractScalar: json_extract_segments("JSONExtractString", quoted_index=False),
exp.JSONPathKey: json_path_key_only_name,
@ -1036,7 +1051,7 @@ class ClickHouse(Dialect):
exp.SHA2: sha256_sql,
exp.UnixToTime: _unix_to_time_sql,
exp.TimestampTrunc: timestamptrunc_sql(zone=True),
exp.Trim: trim_sql,
exp.Trim: lambda self, e: trim_sql(self, e, default_trim_type="BOTH"),
exp.Variance: rename_func("varSamp"),
exp.SchemaCommentProperty: lambda self, e: self.naked_property(e),
exp.Stddev: rename_func("stddevSamp"),
@ -1092,7 +1107,7 @@ class ClickHouse(Dialect):
if not isinstance(expression.parent, exp.Cast):
# StrToDate returns DATEs in other dialects (eg. postgres), so
# this branch aims to improve the transpilation to clickhouse
return f"CAST({strtodate_sql} AS DATE)"
return self.cast_sql(exp.cast(expression, "DATE"))
return strtodate_sql
@ -1201,19 +1216,6 @@ class ClickHouse(Dialect):
),
]
def parameterizedagg_sql(self, expression: exp.ParameterizedAgg) -> str:
params = self.expressions(expression, key="params", flat=True)
return self.func(expression.name, *expression.expressions) + f"({params})"
def anonymousaggfunc_sql(self, expression: exp.AnonymousAggFunc) -> str:
return self.func(expression.name, *expression.expressions)
def combinedaggfunc_sql(self, expression: exp.CombinedAggFunc) -> str:
return self.anonymousaggfunc_sql(expression)
def combinedparameterizedagg_sql(self, expression: exp.CombinedParameterizedAgg) -> str:
return self.parameterizedagg_sql(expression)
def placeholder_sql(self, expression: exp.Placeholder) -> str:
return f"{{{expression.name}: {self.sql(expression, 'kind')}}}"
@ -1294,3 +1296,18 @@ class ClickHouse(Dialect):
is_sql = self.wrap(is_sql)
return is_sql
def in_sql(self, expression: exp.In) -> str:
in_sql = super().in_sql(expression)
if isinstance(expression.parent, exp.Not) and expression.args.get("is_global"):
in_sql = in_sql.replace("GLOBAL IN", "GLOBAL NOT IN", 1)
return in_sql
def not_sql(self, expression: exp.Not) -> str:
if isinstance(expression.this, exp.In) and expression.this.args.get("is_global"):
# let `GLOBAL IN` child interpose `NOT`
return self.sql(expression, "this")
return super().not_sql(expression)

27
sqlglot/dialects/dialect.py
View file

@ -3,6 +3,8 @@ from __future__ import annotations
import importlib
import logging
import typing as t
import sys
from enum import Enum, auto
from functools import reduce
@ -103,6 +105,20 @@ class NormalizationStrategy(str, AutoName):
"""Always case-insensitive, regardless of quotes."""
class Version(int):
def __new__(cls, version_str: t.Optional[str], *args, **kwargs):
if version_str:
parts = version_str.split(".")
parts.extend(["0"] * (3 - len(parts)))
v = int("".join([p.zfill(3) for p in parts]))
else:
# No version defined means we should support the latest engine semantics, so
# the comparison to any specific version should yield that latest is greater
v = sys.maxsize
return super(Version, cls).__new__(cls, v)
class _Dialect(type):
_classes: t.Dict[str, t.Type[Dialect]] = {}
@ -453,6 +469,9 @@ class Dialect(metaclass=_Dialect):
NUMBERS_CAN_BE_UNDERSCORE_SEPARATED = False
"""Whether number literals can include underscores for better readability"""
HEX_STRING_IS_INTEGER_TYPE: bool = False
"""Whether hex strings such as x'CC' evaluate to integer or binary/blob type"""
REGEXP_EXTRACT_DEFAULT_GROUP = 0
"""The default value for the capturing group."""
@ -1002,6 +1021,10 @@ class Dialect(metaclass=_Dialect):
def generator(self, **opts) -> Generator:
return self.generator_class(dialect=self, **opts)
@property
def version(self) -> Version:
return Version(self.settings.get("version", None))
DialectType = t.Union[str, Dialect, t.Type[Dialect], None]
@ -1384,9 +1407,9 @@ def count_if_to_sum(self: Generator, expression: exp.CountIf) -> str:
return self.func("sum", exp.func("if", cond, 1, 0))
def trim_sql(self: Generator, expression: exp.Trim) -> str:
def trim_sql(self: Generator, expression: exp.Trim, default_trim_type: str = "") -> str:
target = self.sql(expression, "this")
trim_type = self.sql(expression, "position")
trim_type = self.sql(expression, "position") or default_trim_type
remove_chars = self.sql(expression, "expression")
collation = self.sql(expression, "collation")

32
sqlglot/dialects/duckdb.py
View file

@ -8,11 +8,13 @@ from sqlglot.dialects.dialect import (
Dialect,
JSON_EXTRACT_TYPE,
NormalizationStrategy,
Version,
approx_count_distinct_sql,
arrow_json_extract_sql,
binary_from_function,
bool_xor_sql,
build_default_decimal_type,
count_if_to_sum,
date_trunc_to_time,
datestrtodate_sql,
no_datetime_sql,
@ -75,7 +77,10 @@ def _date_delta_sql(self: DuckDB.Generator, expression: DATETIME_DELTA) -> str:
this = exp.cast(this, to_type) if to_type else this
return f"{self.sql(this)} {op} {self.sql(exp.Interval(this=expression.expression, unit=unit))}"
expr = expression.expression
interval = expr if isinstance(expr, exp.Interval) else exp.Interval(this=expr, unit=unit)
return f"{self.sql(this)} {op} {self.sql(interval)}"
# BigQuery -> DuckDB conversion for the DATE function
@ -279,6 +284,7 @@ class DuckDB(Dialect):
SUPPORTS_ORDER_BY_ALL = True
SUPPORTS_FIXED_SIZE_ARRAYS = True
STRICT_JSON_PATH_SYNTAX = False
NUMBERS_CAN_BE_UNDERSCORE_SEPARATED = True
# https://duckdb.org/docs/sql/introduction.html#creating-a-new-table
NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
@ -373,6 +379,7 @@ class DuckDB(Dialect):
"DECODE": lambda args: exp.Decode(
this=seq_get(args, 0), charset=exp.Literal.string("utf-8")
),
"EDITDIST3": exp.Levenshtein.from_arg_list,
"ENCODE": lambda args: exp.Encode(
this=seq_get(args, 0), charset=exp.Literal.string("utf-8")
),
@ -380,6 +387,7 @@ class DuckDB(Dialect):
"EPOCH_MS": lambda args: exp.UnixToTime(
this=seq_get(args, 0), scale=exp.UnixToTime.MILLIS
),
"GENERATE_SERIES": _build_generate_series(),
"JSON": exp.ParseJSON.from_arg_list,
"JSON_EXTRACT_PATH": parser.build_extract_json_with_path(exp.JSONExtract),
"JSON_EXTRACT_STRING": parser.build_extract_json_with_path(exp.JSONExtractScalar),
@ -391,6 +399,7 @@ class DuckDB(Dialect):
"MAKE_TIMESTAMP": _build_make_timestamp,
"QUANTILE_CONT": exp.PercentileCont.from_arg_list,
"QUANTILE_DISC": exp.PercentileDisc.from_arg_list,
"RANGE": _build_generate_series(end_exclusive=True),
"REGEXP_EXTRACT": build_regexp_extract(exp.RegexpExtract),
"REGEXP_EXTRACT_ALL": build_regexp_extract(exp.RegexpExtractAll),
"REGEXP_MATCHES": exp.RegexpLike.from_arg_list,
@ -400,6 +409,7 @@ class DuckDB(Dialect):
replacement=seq_get(args, 2),
modifiers=seq_get(args, 3),
),
"SHA256": lambda args: exp.SHA2(this=seq_get(args, 0), length=exp.Literal.number(256)),
"STRFTIME": build_formatted_time(exp.TimeToStr, "duckdb"),
"STRING_SPLIT": exp.Split.from_arg_list,
"STRING_SPLIT_REGEX": exp.RegexpSplit.from_arg_list,
@ -412,9 +422,6 @@ class DuckDB(Dialect):
"TO_TIMESTAMP": exp.UnixToTime.from_arg_list,
"UNNEST": exp.Explode.from_arg_list,
"XOR": binary_from_function(exp.BitwiseXor),
"GENERATE_SERIES": _build_generate_series(),
"RANGE": _build_generate_series(end_exclusive=True),
"EDITDIST3": exp.Levenshtein.from_arg_list,
}
FUNCTIONS.pop("DATE_SUB")
@ -470,7 +477,9 @@ class DuckDB(Dialect):
self, this: t.Optional[exp.Expression] = None
) -> t.Optional[exp.Expression]:
bracket = super()._parse_bracket(this)
if isinstance(bracket, exp.Bracket):
if self.dialect.version < Version("1.2.0") and isinstance(bracket, exp.Bracket):
# https://duckdb.org/2025/02/05/announcing-duckdb-120.html#breaking-changes
bracket.set("returns_list_for_maps", True)
return bracket
@ -574,6 +583,7 @@ class DuckDB(Dialect):
exp.Encode: lambda self, e: encode_decode_sql(self, e, "ENCODE", replace=False),
exp.GenerateDateArray: _generate_datetime_array_sql,
exp.GenerateTimestampArray: _generate_datetime_array_sql,
exp.HexString: lambda self, e: self.hexstring_sql(e, binary_function_repr="FROM_HEX"),
exp.Explode: rename_func("UNNEST"),
exp.IntDiv: lambda self, e: self.binary(e, "//"),
exp.IsInf: rename_func("ISINF"),
@ -682,6 +692,7 @@ class DuckDB(Dialect):
exp.DataType.Type.CHAR: "TEXT",
exp.DataType.Type.DATETIME: "TIMESTAMP",
exp.DataType.Type.FLOAT: "REAL",
exp.DataType.Type.JSONB: "JSON",
exp.DataType.Type.NCHAR: "TEXT",
exp.DataType.Type.NVARCHAR: "TEXT",
exp.DataType.Type.UINT: "UINTEGER",
@ -894,7 +905,18 @@ class DuckDB(Dialect):
return self.function_fallback_sql(expression)
def countif_sql(self, expression: exp.CountIf) -> str:
if self.dialect.version >= Version("1.2"):
return self.function_fallback_sql(expression)
# https://github.com/tobymao/sqlglot/pull/4749
return count_if_to_sum(self, expression)
def bracket_sql(self, expression: exp.Bracket) -> str:
if self.dialect.version >= Version("1.2"):
return super().bracket_sql(expression)
# https://duckdb.org/2025/02/05/announcing-duckdb-120.html#breaking-changes
this = expression.this
if isinstance(this, exp.Array):
this.replace(exp.paren(this))

19
sqlglot/dialects/mysql.py
View file

@ -299,6 +299,7 @@ class MySQL(Dialect):
"CONVERT_TZ": lambda args: exp.ConvertTimezone(
source_tz=seq_get(args, 1), target_tz=seq_get(args, 2), timestamp=seq_get(args, 0)
),
"CURDATE": exp.CurrentDate.from_arg_list,
"DATE": lambda args: exp.TsOrDsToDate(this=seq_get(args, 0)),
"DATE_ADD": build_date_delta_with_interval(exp.DateAdd),
"DATE_FORMAT": build_formatted_time(exp.TimeToStr, "mysql"),
@ -438,6 +439,11 @@ class MySQL(Dialect):
"MODIFY": lambda self: self._parse_alter_table_alter(),
}
ALTER_ALTER_PARSERS = {
**parser.Parser.ALTER_ALTER_PARSERS,
"INDEX": lambda self: self._parse_alter_table_alter_index(),
}
SCHEMA_UNNAMED_CONSTRAINTS = {
*parser.Parser.SCHEMA_UNNAMED_CONSTRAINTS,
"FULLTEXT",
@ -692,6 +698,18 @@ class MySQL(Dialect):
on_condition=self._parse_on_condition(),
)
def _parse_alter_table_alter_index(self) -> exp.AlterIndex:
index = self._parse_field(any_token=True)
if self._match_text_seq("VISIBLE"):
visible = True
elif self._match_text_seq("INVISIBLE"):
visible = False
else:
visible = None
return self.expression(exp.AlterIndex, this=index, visible=visible)
class Generator(generator.Generator):
INTERVAL_ALLOWS_PLURAL_FORM = False
LOCKING_READS_SUPPORTED = True
@ -792,6 +810,7 @@ class MySQL(Dialect):
exp.DataType.Type.USMALLINT: "SMALLINT",
exp.DataType.Type.UTINYINT: "TINYINT",
exp.DataType.Type.UDECIMAL: "DECIMAL",
exp.DataType.Type.UDOUBLE: "DOUBLE",
}
TIMESTAMP_TYPE_MAPPING = {

2
sqlglot/dialects/postgres.py
View file

@ -650,7 +650,7 @@ class Postgres(Dialect):
if isinstance(expression.parent, (exp.From, exp.Join)):
generate_series = (
exp.select("value::date")
.from_(generate_series.as_("value"))
.from_(exp.Table(this=generate_series).as_("_t", table=["value"]))
.subquery(expression.args.get("alias") or "_unnested_generate_series")
)
return self.sql(generate_series)

2
sqlglot/dialects/presto.py
View file

@ -222,6 +222,7 @@ class Presto(Dialect):
}
class Tokenizer(tokens.Tokenizer):
HEX_STRINGS = [("x'", "'"), ("X'", "'")]
UNICODE_STRINGS = [
(prefix + q, q)
for q in t.cast(t.List[str], tokens.Tokenizer.QUOTES)
@ -400,7 +401,6 @@ class Presto(Dialect):
exp.If: if_sql(),
exp.ILike: no_ilike_sql,
exp.Initcap: _initcap_sql,
exp.JSONExtract: lambda self, e: self.jsonextract_sql(e),
exp.Last: _first_last_sql,
exp.LastDay: lambda self, e: self.func("LAST_DAY_OF_MONTH", e.this),
exp.Lateral: explode_to_unnest_sql,

48
sqlglot/dialects/snowflake.py
View file

@ -176,19 +176,27 @@ def _date_trunc_to_time(args: t.List) -> exp.DateTrunc | exp.TimestampTrunc:
return trunc
def _unqualify_unpivot_columns(expression: exp.Expression) -> exp.Expression:
def _unqualify_pivot_columns(expression: exp.Expression) -> exp.Expression:
"""
Snowflake doesn't allow columns referenced in UNPIVOT to be qualified,
so we need to unqualify them.
so we need to unqualify them. Same goes for ANY ORDER BY <column>.
Example:
>>> from sqlglot import parse_one
>>> expr = parse_one("SELECT * FROM m_sales UNPIVOT(sales FOR month IN (m_sales.jan, feb, mar, april))")
>>> print(_unqualify_unpivot_columns(expr).sql(dialect="snowflake"))
>>> print(_unqualify_pivot_columns(expr).sql(dialect="snowflake"))
SELECT * FROM m_sales UNPIVOT(sales FOR month IN (jan, feb, mar, april))
"""
if isinstance(expression, exp.Pivot) and expression.unpivot:
if isinstance(expression, exp.Pivot):
if expression.unpivot:
expression = transforms.unqualify_columns(expression)
else:
field = expression.args.get("field")
field_expr = seq_get(field.expressions if field else [], 0)
if isinstance(field_expr, exp.PivotAny):
unqualified_field_expr = transforms.unqualify_columns(field_expr)
t.cast(exp.Expression, field).set("expressions", unqualified_field_expr, 0)
return expression
@ -512,6 +520,8 @@ class Snowflake(Dialect):
}
SHOW_PARSERS = {
"DATABASES": _show_parser("DATABASES"),
"TERSE DATABASES": _show_parser("DATABASES"),
"SCHEMAS": _show_parser("SCHEMAS"),
"TERSE SCHEMAS": _show_parser("SCHEMAS"),
"OBJECTS": _show_parser("OBJECTS"),
@ -531,6 +541,9 @@ class Snowflake(Dialect):
"COLUMNS": _show_parser("COLUMNS"),
"USERS": _show_parser("USERS"),
"TERSE USERS": _show_parser("USERS"),
"FUNCTIONS": _show_parser("FUNCTIONS"),
"PROCEDURES": _show_parser("PROCEDURES"),
"WAREHOUSES": _show_parser("WAREHOUSES"),
}
CONSTRAINT_PARSERS = {
@ -565,6 +578,16 @@ class Snowflake(Dialect):
),
}
def _parse_use(self) -> exp.Use:
if self._match_text_seq("SECONDARY", "ROLES"):
this = self._match_texts(("ALL", "NONE")) and exp.var(self._prev.text.upper())
roles = None if this else self._parse_csv(lambda: self._parse_table(schema=False))
return self.expression(
exp.Use, kind="SECONDARY ROLES", this=this, expressions=roles
)
return super()._parse_use()
def _negate_range(
self, this: t.Optional[exp.Expression] = None
) -> t.Optional[exp.Expression]:
@ -749,6 +772,14 @@ class Snowflake(Dialect):
if self._match(TokenType.IN):
if self._match_text_seq("ACCOUNT"):
scope_kind = "ACCOUNT"
elif self._match_text_seq("CLASS"):
scope_kind = "CLASS"
scope = self._parse_table_parts()
elif self._match_text_seq("APPLICATION"):
scope_kind = "APPLICATION"
if self._match_text_seq("PACKAGE"):
scope_kind += " PACKAGE"
scope = self._parse_table_parts()
elif self._match_set(self.DB_CREATABLES):
scope_kind = self._prev.text.upper()
if self._curr:
@ -769,6 +800,8 @@ class Snowflake(Dialect):
"starts_with": self._match_text_seq("STARTS", "WITH") and self._parse_string(),
"limit": self._parse_limit(),
"from": self._parse_string() if self._match(TokenType.FROM) else None,
"privileges": self._match_text_seq("WITH", "PRIVILEGES")
and self._parse_csv(lambda: self._parse_var(any_token=True, upper=True)),
},
)
@ -942,7 +975,7 @@ class Snowflake(Dialect):
exp.PercentileDisc: transforms.preprocess(
[transforms.add_within_group_for_percentiles]
),
exp.Pivot: transforms.preprocess([_unqualify_unpivot_columns]),
exp.Pivot: transforms.preprocess([_unqualify_pivot_columns]),
exp.RegexpExtract: _regexpextract_sql,
exp.RegexpExtractAll: _regexpextract_sql,
exp.RegexpILike: _regexpilike_sql,
@ -1139,7 +1172,10 @@ class Snowflake(Dialect):
if from_:
from_ = f" FROM {from_}"
return f"SHOW {terse}{expression.name}{history}{like}{scope_kind}{scope}{starts_with}{limit}{from_}"
privileges = self.expressions(expression, key="privileges", flat=True)
privileges = f" WITH PRIVILEGES {privileges}" if privileges else ""
return f"SHOW {terse}{expression.name}{history}{like}{scope_kind}{scope}{starts_with}{limit}{from_}{privileges}"
def describe_sql(self, expression: exp.Describe) -> str:
# Default to table if kind is unknown

1
sqlglot/dialects/sqlite.py
View file

@ -140,6 +140,7 @@ class SQLite(Dialect):
SUPPORTS_TO_NUMBER = False
EXCEPT_INTERSECT_SUPPORT_ALL_CLAUSE = False
SUPPORTS_MEDIAN = False
JSON_KEY_VALUE_PAIR_SEP = ","
SUPPORTED_JSON_PATH_PARTS = {
exp.JSONPathKey,

14
sqlglot/dialects/trino.py
View file

@ -11,9 +11,6 @@ class Trino(Presto):
SUPPORTS_USER_DEFINED_TYPES = False
LOG_BASE_FIRST = True
class Tokenizer(Presto.Tokenizer):
HEX_STRINGS = [("X'", "'")]
class Parser(Presto.Parser):
FUNCTION_PARSERS = {
**Presto.Parser.FUNCTION_PARSERS,
@ -56,6 +53,7 @@ class Trino(Presto):
option=self._parse_var_from_options(self.JSON_QUERY_OPTIONS, raise_unmatched=False),
json_query=True,
quote=self._parse_json_query_quote(),
on_condition=self._parse_on_condition(),
)
class Generator(Presto.Generator):
@ -73,7 +71,6 @@ class Trino(Presto):
exp.Merge: merge_without_target_sql,
exp.TimeStrToTime: lambda self, e: timestrtotime_sql(self, e, include_precision=True),
exp.Trim: trim_sql,
exp.JSONExtract: lambda self, e: self.jsonextract_sql(e),
}
SUPPORTED_JSON_PATH_PARTS = {
@ -93,7 +90,14 @@ class Trino(Presto):
quote = self.sql(expression, "quote")
quote = f" {quote}" if quote else ""
return self.func("JSON_QUERY", expression.this, json_path + option + quote)
on_condition = self.sql(expression, "on_condition")
on_condition = f" {on_condition}" if on_condition else ""
return self.func(
"JSON_QUERY",
expression.this,
json_path + option + quote + on_condition,
)
def groupconcat_sql(self, expression: exp.GroupConcat) -> str:
this = expression.this

14
sqlglot/dialects/tsql.py
View file

@ -1282,20 +1282,6 @@ class TSQL(Dialect):
expression.this.set("catalog", None)
return super().drop_sql(expression)
def declare_sql(self, expression: exp.Declare) -> str:
return f"DECLARE {self.expressions(expression, flat=True)}"
def declareitem_sql(self, expression: exp.DeclareItem) -> str:
variable = self.sql(expression, "this")
default = self.sql(expression, "default")
default = f" = {default}" if default else ""
kind = self.sql(expression, "kind")
if isinstance(expression.args.get("kind"), exp.Schema):
kind = f"TABLE {kind}"
return f"{variable} AS {kind}{default}"
def options_modifier(self, expression: exp.Expression) -> str:
options = self.expressions(expression, key="options")
return f" OPTION{self.wrap(options)}" if options else ""

54
sqlglot/diff.py
View file

@ -68,7 +68,6 @@ def diff(
target: exp.Expression,
matchings: t.List[t.Tuple[exp.Expression, exp.Expression]] | None = None,
delta_only: bool = False,
copy: bool = True,
**kwargs: t.Any,
) -> t.List[Edit]:
"""
@ -97,9 +96,6 @@ def diff(
Note: expression references in this list must refer to the same node objects that are
referenced in the source / target trees.
delta_only: excludes all `Keep` nodes from the diff.
copy: whether to copy the input expressions.
Note: if this is set to false, the caller must ensure that there are no shared references
in the two trees, otherwise the diffing algorithm may produce unexpected behavior.
kwargs: additional arguments to pass to the ChangeDistiller instance.
Returns:
@ -108,43 +104,53 @@ def diff(
expression tree into the target one.
"""
matchings = matchings or []
matching_ids = {id(n) for pair in matchings for n in pair}
def compute_node_mappings(
original: exp.Expression, copy: exp.Expression
old_nodes: tuple[exp.Expression, ...], new_nodes: tuple[exp.Expression, ...]
) -> t.Dict[int, exp.Expression]:
node_mapping = {}
for old_node, new_node in zip(
reversed(tuple(original.walk())), reversed(tuple(copy.walk()))
):
# We cache the hash of each new node here to speed up equality comparisons. If the input
# trees aren't copied, these hashes will be evicted before returning the edit script.
for old_node, new_node in zip(reversed(old_nodes), reversed(new_nodes)):
new_node._hash = hash(new_node)
old_node_id = id(old_node)
if old_node_id in matching_ids:
node_mapping[old_node_id] = new_node
node_mapping[id(old_node)] = new_node
return node_mapping
# if the source and target have any shared objects, that means there's an issue with the ast
# the algorithm won't work because the parent / hierarchies will be inaccurate
source_nodes = tuple(source.walk())
target_nodes = tuple(target.walk())
source_ids = {id(n) for n in source_nodes}
target_ids = {id(n) for n in target_nodes}
copy = (
len(source_nodes) != len(source_ids)
or len(target_nodes) != len(target_ids)
or source_ids & target_ids
)
source_copy = source.copy() if copy else source
target_copy = target.copy() if copy else target
node_mappings = {
**compute_node_mappings(source, source_copy),
**compute_node_mappings(target, target_copy),
}
matchings_copy = [(node_mappings[id(s)], node_mappings[id(t)]) for s, t in matchings]
try:
# We cache the hash of each new node here to speed up equality comparisons. If the input
# trees aren't copied, these hashes will be evicted before returning the edit script.
if copy and matchings:
source_mapping = compute_node_mappings(source_nodes, tuple(source_copy.walk()))
target_mapping = compute_node_mappings(target_nodes, tuple(target_copy.walk()))
matchings = [(source_mapping[id(s)], target_mapping[id(t)]) for s, t in matchings]
else:
for node in chain(reversed(source_nodes), reversed(target_nodes)):
node._hash = hash(node)
edit_script = ChangeDistiller(**kwargs).diff(
source_copy,
target_copy,
matchings=matchings_copy,
matchings=matchings,
delta_only=delta_only,
)
finally:
if not copy:
for node in chain(source.walk(), target.walk()):
for node in chain(source_nodes, target_nodes):
node._hash = None
return edit_script
@ -186,8 +192,6 @@ class ChangeDistiller:
) -> t.List[Edit]:
matchings = matchings or []
pre_matched_nodes = {id(s): id(t) for s, t in matchings}
if len({n for pair in pre_matched_nodes.items() for n in pair}) != 2 * len(matchings):
raise ValueError("Each node can be referenced at most once in the list of matchings")
self._source = source
self._target = target

64
sqlglot/expressions.py
View file

@ -1557,6 +1557,7 @@ class Show(Expression):
"log": False,
"position": False,
"types": False,
"privileges": False,
}
@ -1568,8 +1569,12 @@ class CharacterSet(Expression):
arg_types = {"this": True, "default": False}
class RecursiveWithSearch(Expression):
arg_types = {"kind": True, "this": True, "expression": True, "using": False}
class With(Expression):
arg_types = {"expressions": True, "recursive": False}
arg_types = {"expressions": True, "recursive": False, "search": False}
@property
def recursive(self) -> bool:
@ -1608,7 +1613,7 @@ class BitString(Condition):
class HexString(Condition):
pass
arg_types = {"this": True, "is_integer": False}
class ByteString(Condition):
@ -1698,9 +1703,15 @@ class AlterColumn(Expression):
"drop": False,
"comment": False,
"allow_null": False,
"visible": False,
}
# https://dev.mysql.com/doc/refman/8.0/en/invisible-indexes.html
class AlterIndex(Expression):
arg_types = {"this": True, "visible": True}
# https://docs.aws.amazon.com/redshift/latest/dg/r_ALTER_TABLE.html
class AlterDistStyle(Expression):
pass
@ -2352,8 +2363,7 @@ class Fetch(Expression):
arg_types = {
"direction": False,
"count": False,
"percent": False,
"with_ties": False,
"limit_options": False,
}
@ -2395,7 +2405,21 @@ class Lambda(Expression):
class Limit(Expression):
arg_types = {"this": False, "expression": True, "offset": False, "expressions": False}
arg_types = {
"this": False,
"expression": True,
"offset": False,
"limit_options": False,
"expressions": False,
}
class LimitOptions(Expression):
arg_types = {
"percent": False,
"rows": False,
"with_ties": False,
}
class Literal(Condition):
@ -4475,6 +4499,7 @@ class DataType(Expression):
UINT256 = auto()
UMEDIUMINT = auto()
UDECIMAL = auto()
UDOUBLE = auto()
UNION = auto()
UNKNOWN = auto() # Sentinel value, useful for type annotation
USERDEFINED = "USER-DEFINED"
@ -4558,6 +4583,7 @@ class DataType(Expression):
Type.MONEY,
Type.SMALLMONEY,
Type.UDECIMAL,
Type.UDOUBLE,
}
NUMERIC_TYPES = {
@ -4827,14 +4853,6 @@ class Connector(Binary):
pass
class And(Connector):
pass
class Or(Connector):
pass
class BitwiseAnd(Binary):
pass
@ -5288,11 +5306,11 @@ class AnonymousAggFunc(AggFunc):
# https://clickhouse.com/docs/en/sql-reference/aggregate-functions/combinators
class CombinedAggFunc(AnonymousAggFunc):
arg_types = {"this": True, "expressions": False, "parts": True}
arg_types = {"this": True, "expressions": False}
class CombinedParameterizedAgg(ParameterizedAgg):
arg_types = {"this": True, "expressions": True, "params": True, "parts": True}
arg_types = {"this": True, "expressions": True, "params": True}
# https://docs.snowflake.com/en/sql-reference/functions/hll
@ -5558,6 +5576,11 @@ class TryCast(Cast):
pass
# https://clickhouse.com/docs/sql-reference/data-types/newjson#reading-json-paths-as-sub-columns
class JSONCast(Cast):
pass
class Try(Func):
pass
@ -5959,6 +5982,14 @@ class LowerHex(Hex):
pass
class And(Connector, Func):
pass
class Or(Connector, Func):
pass
class Xor(Connector, Func):
arg_types = {"this": False, "expression": False, "expressions": False}
@ -6181,6 +6212,7 @@ class JSONExtract(Binary, Func):
"json_query": False,
"option": False,
"quote": False,
"on_condition": False,
}
_sql_names = ["JSON_EXTRACT"]
is_var_len_args = True
@ -6820,7 +6852,7 @@ class Year(Func):
class Use(Expression):
arg_types = {"this": True, "kind": False}
arg_types = {"this": False, "expressions": False, "kind": False}
class Merge(DML):

110
sqlglot/generator.py
View file

@ -1238,8 +1238,10 @@ class Generator(metaclass=_Generator):
if self.CTE_RECURSIVE_KEYWORD_REQUIRED and expression.args.get("recursive")
else ""
)
search = self.sql(expression, "search")
search = f" {search}" if search else ""
return f"WITH {recursive}{sql}"
return f"WITH {recursive}{sql}{search}"
def cte_sql(self, expression: exp.CTE) -> str:
alias = expression.args.get("alias")
@ -1276,12 +1278,31 @@ class Generator(metaclass=_Generator):
return f"{self.dialect.BIT_START}{this}{self.dialect.BIT_END}"
return f"{int(this, 2)}"
def hexstring_sql(self, expression: exp.HexString) -> str:
def hexstring_sql(
self, expression: exp.HexString, binary_function_repr: t.Optional[str] = None
) -> str:
this = self.sql(expression, "this")
if self.dialect.HEX_START:
return f"{self.dialect.HEX_START}{this}{self.dialect.HEX_END}"
is_integer_type = expression.args.get("is_integer")
if (is_integer_type and not self.dialect.HEX_STRING_IS_INTEGER_TYPE) or (
not self.dialect.HEX_START and not binary_function_repr
):
# Integer representation will be returned if:
# - The read dialect treats the hex value as integer literal but not the write
# - The transpilation is not supported (write dialect hasn't set HEX_START or the param flag)
return f"{int(this, 16)}"
if not is_integer_type:
# Read dialect treats the hex value as BINARY/BLOB
if binary_function_repr:
# The write dialect supports the transpilation to its equivalent BINARY/BLOB
return self.func(binary_function_repr, exp.Literal.string(this))
if self.dialect.HEX_STRING_IS_INTEGER_TYPE:
# The write dialect does not support the transpilation, it'll treat the hex value as INTEGER
self.unsupported("Unsupported transpilation from BINARY/BLOB hex string")
return f"{self.dialect.HEX_START}{this}{self.dialect.HEX_END}"
def bytestring_sql(self, expression: exp.ByteString) -> str:
this = self.sql(expression, "this")
if self.dialect.BYTE_START:
@ -1467,10 +1488,17 @@ class Generator(metaclass=_Generator):
direction = f" {direction}" if direction else ""
count = self.sql(expression, "count")
count = f" {count}" if count else ""
if expression.args.get("percent"):
count = f"{count} PERCENT"
with_ties_or_only = "WITH TIES" if expression.args.get("with_ties") else "ONLY"
return f"{self.seg('FETCH')}{direction}{count} ROWS {with_ties_or_only}"
limit_options = self.sql(expression, "limit_options")
limit_options = f"{limit_options}" if limit_options else " ROWS ONLY"
return f"{self.seg('FETCH')}{direction}{count}{limit_options}"
def limitoptions_sql(self, expression: exp.LimitOptions) -> str:
percent = " PERCENT" if expression.args.get("percent") else ""
rows = " ROWS" if expression.args.get("rows") else ""
with_ties = " WITH TIES" if expression.args.get("with_ties") else ""
if not with_ties and rows:
with_ties = " ONLY"
return f"{percent}{rows}{with_ties}"
def filter_sql(self, expression: exp.Filter) -> str:
if self.AGGREGATE_FILTER_SUPPORTED:
@ -2261,9 +2289,10 @@ class Generator(metaclass=_Generator):
args_sql = ", ".join(self.sql(e) for e in args)
args_sql = f"({args_sql})" if top and any(not e.is_number for e in args) else args_sql
expressions = self.expressions(expression, flat=True)
limit_options = self.sql(expression, "limit_options")
expressions = f" BY {expressions}" if expressions else ""
return f"{this}{self.seg('TOP' if top else 'LIMIT')} {args_sql}{expressions}"
return f"{this}{self.seg('TOP' if top else 'LIMIT')} {args_sql}{limit_options}{expressions}"
def offset_sql(self, expression: exp.Offset) -> str:
this = self.sql(expression, "this")
@ -2284,9 +2313,7 @@ class Generator(metaclass=_Generator):
return f"{global_}{kind}{this}{expressions}{collate}"
def set_sql(self, expression: exp.Set) -> str:
expressions = (
f" {self.expressions(expression, flat=True)}" if expression.expressions else ""
)
expressions = f" {self.expressions(expression, flat=True)}"
tag = " TAG" if expression.args.get("tag") else ""
return f"{'UNSET' if expression.args.get('unset') else 'SET'}{tag}{expressions}"
@ -3265,6 +3292,10 @@ class Generator(metaclass=_Generator):
if comment:
return f"ALTER COLUMN {this} COMMENT {comment}"
visible = expression.args.get("visible")
if visible:
return f"ALTER COLUMN {this} SET {visible}"
allow_null = expression.args.get("allow_null")
drop = expression.args.get("drop")
@ -3277,6 +3308,14 @@ class Generator(metaclass=_Generator):
return f"ALTER COLUMN {this} DROP DEFAULT"
def alterindex_sql(self, expression: exp.AlterIndex) -> str:
this = self.sql(expression, "this")
visible = expression.args.get("visible")
visible_sql = "VISIBLE" if visible else "INVISIBLE"
return f"ALTER INDEX {this} {visible_sql}"
def alterdiststyle_sql(self, expression: exp.AlterDistStyle) -> str:
this = self.sql(expression, "this")
if not isinstance(expression.this, exp.Var):
@ -3499,6 +3538,9 @@ class Generator(metaclass=_Generator):
def trycast_sql(self, expression: exp.TryCast) -> str:
return self.cast_sql(expression, safe_prefix="TRY_")
def jsoncast_sql(self, expression: exp.JSONCast) -> str:
return self.cast_sql(expression)
def try_sql(self, expression: exp.Try) -> str:
if not self.TRY_SUPPORTED:
self.unsupported("Unsupported TRY function")
@ -3523,7 +3565,7 @@ class Generator(metaclass=_Generator):
def use_sql(self, expression: exp.Use) -> str:
kind = self.sql(expression, "kind")
kind = f" {kind}" if kind else ""
this = self.sql(expression, "this")
this = self.sql(expression, "this") or self.expressions(expression, flat=True)
this = f" {this}" if this else ""
return f"USE{kind}{this}"
@ -4762,3 +4804,45 @@ class Generator(metaclass=_Generator):
connection = f"WITH CONNECTION {connection} " if connection else ""
options = self.sql(expression, "options")
return f"EXPORT DATA {connection}{options} AS {this}"
def declare_sql(self, expression: exp.Declare) -> str:
return f"DECLARE {self.expressions(expression, flat=True)}"
def declareitem_sql(self, expression: exp.DeclareItem) -> str:
variable = self.sql(expression, "this")
default = self.sql(expression, "default")
default = f" = {default}" if default else ""
kind = self.sql(expression, "kind")
if isinstance(expression.args.get("kind"), exp.Schema):
kind = f"TABLE {kind}"
return f"{variable} AS {kind}{default}"
def recursivewithsearch_sql(self, expression: exp.RecursiveWithSearch) -> str:
kind = self.sql(expression, "kind")
this = self.sql(expression, "this")
set = self.sql(expression, "expression")
using = self.sql(expression, "using")
using = f" USING {using}" if using else ""
kind_sql = kind if kind == "CYCLE" else f"SEARCH {kind} FIRST BY"
return f"{kind_sql} {this} SET {set}{using}"
def parameterizedagg_sql(self, expression: exp.ParameterizedAgg) -> str:
params = self.expressions(expression, key="params", flat=True)
return self.func(expression.name, *expression.expressions) + f"({params})"
def anonymousaggfunc_sql(self, expression: exp.AnonymousAggFunc) -> str:
return self.func(expression.name, *expression.expressions)
def combinedaggfunc_sql(self, expression: exp.CombinedAggFunc) -> str:
return self.anonymousaggfunc_sql(expression)
def combinedparameterizedagg_sql(self, expression: exp.CombinedParameterizedAgg) -> str:
return self.parameterizedagg_sql(expression)
def show_sql(self, expression: exp.Show) -> str:
self.unsupported("Unsupported SHOW statement")
return ""

30
sqlglot/optimizer/annotate_types.py
View file

@ -195,7 +195,7 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
# Maps an exp.SetOperation's id (e.g. UNION) to its projection types. This is computed if the
# exp.SetOperation is the expression of a scope source, as selecting from it multiple times
# would reprocess the entire subtree to coerce the types of its operands' projections
self._setop_column_types: t.Dict[int, t.Dict[str, exp.DataType.Type]] = {}
self._setop_column_types: t.Dict[int, t.Dict[str, exp.DataType | exp.DataType.Type]] = {}
def _set_type(
self, expression: exp.Expression, target_type: t.Optional[exp.DataType | exp.DataType.Type]
@ -312,18 +312,32 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
def _maybe_coerce(
self, type1: exp.DataType | exp.DataType.Type, type2: exp.DataType | exp.DataType.Type
) -> exp.DataType.Type:
type1_value = type1.this if isinstance(type1, exp.DataType) else type1
type2_value = type2.this if isinstance(type2, exp.DataType) else type2
) -> exp.DataType | exp.DataType.Type:
"""
Returns type2 if type1 can be coerced into it, otherwise type1.
If either type is parameterized (e.g. DECIMAL(18, 2) contains two parameters),
we assume type1 does not coerce into type2, so we also return it in this case.
"""
if isinstance(type1, exp.DataType):
if type1.expressions:
return type1
type1_value = type1.this
else:
type1_value = type1
if isinstance(type2, exp.DataType):
if type2.expressions:
return type1
type2_value = type2.this
else:
type2_value = type2
# We propagate the UNKNOWN type upwards if found
if exp.DataType.Type.UNKNOWN in (type1_value, type2_value):
return exp.DataType.Type.UNKNOWN
return t.cast(
exp.DataType.Type,
type2_value if type2_value in self.coerces_to.get(type1_value, {}) else type1_value,
)
return type2_value if type2_value in self.coerces_to.get(type1_value, {}) else type1_value
def _annotate_binary(self, expression: B) -> B:
self._annotate_args(expression)

44
sqlglot/optimizer/qualify_columns.py
View file

@ -102,7 +102,10 @@ def qualify_columns(
qualify_outputs(scope)
_expand_group_by(scope, dialect)
_expand_order_by(scope, resolver)
# DISTINCT ON and ORDER BY follow the same rules (tested in DuckDB, Postgres, ClickHouse)
# https://www.postgresql.org/docs/current/sql-select.html#SQL-DISTINCT
_expand_order_by_and_distinct_on(scope, resolver)
if dialect == "bigquery":
annotator.annotate_scope(scope)
@ -359,35 +362,40 @@ def _expand_group_by(scope: Scope, dialect: DialectType) -> None:
expression.set("group", group)
def _expand_order_by(scope: Scope, resolver: Resolver) -> None:
order = scope.expression.args.get("order")
if not order:
return
def _expand_order_by_and_distinct_on(scope: Scope, resolver: Resolver) -> None:
for modifier_key in ("order", "distinct"):
modifier = scope.expression.args.get(modifier_key)
if isinstance(modifier, exp.Distinct):
modifier = modifier.args.get("on")
ordereds = order.expressions
for ordered, new_expression in zip(
ordereds,
if not isinstance(modifier, exp.Expression):
continue
modifier_expressions = modifier.expressions
if modifier_key == "order":
modifier_expressions = [ordered.this for ordered in modifier_expressions]
for original, expanded in zip(
modifier_expressions,
_expand_positional_references(
scope, (o.this for o in ordereds), resolver.schema.dialect, alias=True
scope, modifier_expressions, resolver.schema.dialect, alias=True
),
):
for agg in ordered.find_all(exp.AggFunc):
for agg in original.find_all(exp.AggFunc):
for col in agg.find_all(exp.Column):
if not col.table:
col.set("table", resolver.get_table(col.name))
ordered.set("this", new_expression)
original.replace(expanded)
if scope.expression.args.get("group"):
selects = {s.this: exp.column(s.alias_or_name) for s in scope.expression.selects}
for ordered in ordereds:
ordered = ordered.this
ordered.replace(
exp.to_identifier(_select_by_pos(scope, ordered).alias)
if ordered.is_int
else selects.get(ordered, ordered)
for expression in modifier_expressions:
expression.replace(
exp.to_identifier(_select_by_pos(scope, expression).alias)
if expression.is_int
else selects.get(expression, expression)
)

14
sqlglot/optimizer/qualify_tables.py
View file

@ -97,12 +97,16 @@ def qualify_tables(
source.alias
)
_qualify(source)
if pivots:
if not pivots[0].alias:
pivot_alias = next_alias_name()
pivots[0].set("alias", exp.TableAlias(this=exp.to_identifier(pivot_alias)))
if pivots and not pivots[0].alias:
pivots[0].set(
"alias", exp.TableAlias(this=exp.to_identifier(next_alias_name()))
)
# This case corresponds to a pivoted CTE, we don't want to qualify that
if isinstance(scope.sources.get(source.alias_or_name), Scope):
continue
_qualify(source)
if infer_csv_schemas and schema and isinstance(source.this, exp.ReadCSV):
with csv_reader(source.this) as reader:

13
sqlglot/optimizer/scope.py
View file

@ -282,7 +282,14 @@ class Scope:
self._columns = []
for column in columns + external_columns:
ancestor = column.find_ancestor(
exp.Select, exp.Qualify, exp.Order, exp.Having, exp.Hint, exp.Table, exp.Star
exp.Select,
exp.Qualify,
exp.Order,
exp.Having,
exp.Hint,
exp.Table,
exp.Star,
exp.Distinct,
)
if (
not ancestor
@ -290,9 +297,9 @@ class Scope:
or isinstance(ancestor, exp.Select)
or (isinstance(ancestor, exp.Table) and not isinstance(ancestor.this, exp.Func))
or (
isinstance(ancestor, exp.Order)
isinstance(ancestor, (exp.Order, exp.Distinct))
and (
isinstance(ancestor.parent, exp.Window)
isinstance(ancestor.parent, (exp.Window, exp.WithinGroup))
or column.name not in named_selects
)
)

104
sqlglot/parser.py
View file

@ -318,6 +318,7 @@ class Parser(metaclass=_Parser):
TokenType.FIXEDSTRING,
TokenType.FLOAT,
TokenType.DOUBLE,
TokenType.UDOUBLE,
TokenType.CHAR,
TokenType.NCHAR,
TokenType.VARCHAR,
@ -418,6 +419,7 @@ class Parser(metaclass=_Parser):
TokenType.SMALLINT: TokenType.USMALLINT,
TokenType.TINYINT: TokenType.UTINYINT,
TokenType.DECIMAL: TokenType.UDECIMAL,
TokenType.DOUBLE: TokenType.UDOUBLE,
}
SUBQUERY_PREDICATES = {
@ -733,6 +735,11 @@ class Parser(metaclass=_Parser):
COLUMN_OPERATORS = {
TokenType.DOT: None,
TokenType.DOTCOLON: lambda self, this, to: self.expression(
exp.JSONCast,
this=this,
to=to,
),
TokenType.DCOLON: lambda self, this, to: self.expression(
exp.Cast if self.STRICT_CAST else exp.TryCast,
this=this,
@ -827,11 +834,7 @@ class Parser(metaclass=_Parser):
TokenType.UNCACHE: lambda self: self._parse_uncache(),
TokenType.UNPIVOT: lambda self: self._parse_simplified_pivot(is_unpivot=True),
TokenType.UPDATE: lambda self: self._parse_update(),
TokenType.USE: lambda self: self.expression(
exp.Use,
kind=self._parse_var_from_options(self.USABLES, raise_unmatched=False),
this=self._parse_table(schema=False),
),
TokenType.USE: lambda self: self._parse_use(),
TokenType.SEMICOLON: lambda self: exp.Semicolon(),
}
@ -865,7 +868,11 @@ class Parser(metaclass=_Parser):
NUMERIC_PARSERS = {
TokenType.BIT_STRING: lambda self, token: self.expression(exp.BitString, this=token.text),
TokenType.BYTE_STRING: lambda self, token: self.expression(exp.ByteString, this=token.text),
TokenType.HEX_STRING: lambda self, token: self.expression(exp.HexString, this=token.text),
TokenType.HEX_STRING: lambda self, token: self.expression(
exp.HexString,
this=token.text,
is_integer=self.dialect.HEX_STRING_IS_INTEGER_TYPE or None,
),
TokenType.NUMBER: lambda self, token: self.expression(
exp.Literal, this=token.text, is_string=False
),
@ -1371,6 +1378,8 @@ class Parser(metaclass=_Parser):
OPERATION_MODIFIERS: t.Set[str] = set()
RECURSIVE_CTE_SEARCH_KIND = {"BREADTH", "DEPTH", "CYCLE"}
STRICT_CAST = True
PREFIXED_PIVOT_COLUMNS = False
@ -2967,6 +2976,13 @@ class Parser(metaclass=_Parser):
},
)
def _parse_use(self) -> exp.Use:
return self.expression(
exp.Use,
kind=self._parse_var_from_options(self.USABLES, raise_unmatched=False),
this=self._parse_table(schema=False),
)
def _parse_uncache(self) -> exp.Uncache:
if not self._match(TokenType.TABLE):
self.raise_error("Expecting TABLE after UNCACHE")
@ -3165,6 +3181,24 @@ class Parser(metaclass=_Parser):
return self._parse_set_operations(this) if parse_set_operation else this
def _parse_recursive_with_search(self) -> t.Optional[exp.RecursiveWithSearch]:
self._match_text_seq("SEARCH")
kind = self._match_texts(self.RECURSIVE_CTE_SEARCH_KIND) and self._prev.text.upper()
if not kind:
return None
self._match_text_seq("FIRST", "BY")
return self.expression(
exp.RecursiveWithSearch,
kind=kind,
this=self._parse_id_var(),
expression=self._match_text_seq("SET") and self._parse_id_var(),
using=self._match_text_seq("USING") and self._parse_id_var(),
)
def _parse_with(self, skip_with_token: bool = False) -> t.Optional[exp.With]:
if not skip_with_token and not self._match(TokenType.WITH):
return None
@ -3189,7 +3223,11 @@ class Parser(metaclass=_Parser):
last_comments = self._prev_comments
return self.expression(
exp.With, comments=comments, expressions=expressions, recursive=recursive
exp.With,
comments=comments,
expressions=expressions,
recursive=recursive,
search=self._parse_recursive_with_search(),
)
def _parse_cte(self) -> t.Optional[exp.CTE]:
@ -4205,7 +4243,11 @@ class Parser(metaclass=_Parser):
names = self._pivot_column_names(t.cast(t.List[exp.Expression], expressions))
columns: t.List[exp.Expression] = []
for fld in pivot.args["field"].expressions:
pivot_field_expressions = pivot.args["field"].expressions
# The `PivotAny` expression corresponds to `ANY ORDER BY <column>`; we can't infer in this case.
if not isinstance(seq_get(pivot_field_expressions, 0), exp.PivotAny):
for fld in pivot_field_expressions:
field_name = fld.sql() if self.IDENTIFY_PIVOT_STRINGS else fld.alias_or_name
for name in names:
if self.PREFIXED_PIVOT_COLUMNS:
@ -4413,6 +4455,13 @@ class Parser(metaclass=_Parser):
exp.Ordered, this=this, desc=desc, nulls_first=nulls_first, with_fill=with_fill
)
def _parse_limit_options(self) -> exp.LimitOptions:
percent = self._match(TokenType.PERCENT)
rows = self._match_set((TokenType.ROW, TokenType.ROWS))
self._match_text_seq("ONLY")
with_ties = self._match_text_seq("WITH", "TIES")
return self.expression(exp.LimitOptions, percent=percent, rows=rows, with_ties=with_ties)
def _parse_limit(
self,
this: t.Optional[exp.Expression] = None,
@ -4427,7 +4476,10 @@ class Parser(metaclass=_Parser):
if limit_paren:
self._match_r_paren()
limit_options = self._parse_limit_options()
else:
limit_options = None
expression = self._parse_term()
if self._match(TokenType.COMMA):
@ -4442,6 +4494,7 @@ class Parser(metaclass=_Parser):
expression=expression,
offset=offset,
comments=comments,
limit_options=limit_options,
expressions=self._parse_limit_by(),
)
@ -4452,22 +4505,12 @@ class Parser(metaclass=_Parser):
direction = self._prev.text.upper() if direction else "FIRST"
count = self._parse_field(tokens=self.FETCH_TOKENS)
percent = self._match(TokenType.PERCENT)
self._match_set((TokenType.ROW, TokenType.ROWS))
only = self._match_text_seq("ONLY")
with_ties = self._match_text_seq("WITH", "TIES")
if only and with_ties:
self.raise_error("Cannot specify both ONLY and WITH TIES in FETCH clause")
return self.expression(
exp.Fetch,
direction=direction,
count=count,
percent=percent,
with_ties=with_ties,
limit_options=self._parse_limit_options(),
)
return this
@ -5003,16 +5046,21 @@ class Parser(metaclass=_Parser):
expressions = self._parse_csv(self._parse_equality)
elif is_aggregate:
func_or_ident = self._parse_function(anonymous=True) or self._parse_id_var(
any_token=False, tokens=(TokenType.VAR,)
any_token=False, tokens=(TokenType.VAR, TokenType.ANY)
)
if not func_or_ident or not self._match(TokenType.COMMA):
if not func_or_ident:
return None
expressions = self._parse_csv(
expressions = [func_or_ident]
if self._match(TokenType.COMMA):
expressions.extend(
self._parse_csv(
lambda: self._parse_types(
check_func=check_func, schema=schema, allow_identifiers=allow_identifiers
check_func=check_func,
schema=schema,
allow_identifiers=allow_identifiers,
)
)
)
expressions.insert(0, func_or_ident)
else:
expressions = self._parse_csv(self._parse_type_size)
@ -5281,7 +5329,7 @@ class Parser(metaclass=_Parser):
op_token = self._prev.token_type
op = self.COLUMN_OPERATORS.get(op_token)
if op_token == TokenType.DCOLON:
if op_token in (TokenType.DCOLON, TokenType.DOTCOLON):
field = self._parse_dcolon()
if not field:
self.raise_error("Expected type")
@ -7071,6 +7119,12 @@ class Parser(metaclass=_Parser):
this=column,
allow_null=False,
)
if self._match_text_seq("SET", "VISIBLE"):
return self.expression(exp.AlterColumn, this=column, visible="VISIBLE")
if self._match_text_seq("SET", "INVISIBLE"):
return self.expression(exp.AlterColumn, this=column, visible="INVISIBLE")
self._match_text_seq("SET", "DATA")
self._match_text_seq("TYPE")
return self.expression(

2
sqlglot/tokens.py
View file

@ -37,6 +37,7 @@ class TokenType(AutoName):
DASH = auto()
PLUS = auto()
COLON = auto()
DOTCOLON = auto()
DCOLON = auto()
DQMARK = auto()
SEMICOLON = auto()
@ -123,6 +124,7 @@ class TokenType(AutoName):
UINT256 = auto()
FLOAT = auto()
DOUBLE = auto()
UDOUBLE = auto()
DECIMAL = auto()
DECIMAL32 = auto()
DECIMAL64 = auto()

15
sqlglot/transforms.py
View file

@ -258,7 +258,7 @@ def eliminate_qualify(expression: exp.Expression) -> exp.Expression:
}
select_candidates = exp.Window if expression.is_star else (exp.Window, exp.Column)
for select_candidate in qualify_filters.find_all(select_candidates):
for select_candidate in list(qualify_filters.find_all(select_candidates)):
if isinstance(select_candidate, exp.Window):
if expression_by_alias:
for column in select_candidate.find_all(exp.Column):
@ -907,16 +907,15 @@ def eliminate_join_marks(expression: exp.Expression) -> exp.Expression:
len(marked_column_tables) == 1
), "Columns of only a single table can be marked with (+) in a given binary predicate"
# Add predicate if join already copied, or add join if it is new
join_this = old_joins.get(col.table, query_from).this
new_join = exp.Join(this=join_this, on=join_predicate, kind="LEFT")
# Upsert new_join into new_joins dictionary
new_join_alias_or_name = new_join.alias_or_name
existing_join = new_joins.get(new_join_alias_or_name)
existing_join = new_joins.get(join_this.alias_or_name)
if existing_join:
existing_join.set("on", exp.and_(existing_join.args.get("on"), new_join.args["on"]))
existing_join.set("on", exp.and_(existing_join.args["on"], join_predicate))
else:
new_joins[new_join_alias_or_name] = new_join
new_joins[join_this.alias_or_name] = exp.Join(
this=join_this.copy(), on=join_predicate.copy(), kind="LEFT"
)
# If the parent of the target predicate is a binary node, then it now has only one child
if isinstance(predicate_parent, exp.Binary):

2
sqlglotrs/Cargo.lock generated
View file

@ -503,7 +503,7 @@ dependencies = [
[[package]]
name = "sqlglotrs"
version = "0.3.14"
version = "0.4.0"
dependencies = [
"criterion",
"pyo3",

2
sqlglotrs/Cargo.toml
View file

@ -1,6 +1,6 @@
[package]
name = "sqlglotrs"
version = "0.3.14"
version = "0.4.0"
edition = "2021"
license = "MIT"

11
sqlglotrs/src/tokenizer.rs
View file

@ -142,7 +142,7 @@ impl<'a> TokenizerState<'a> {
break;
}
if !self.settings.white_space.contains_key(&self.current_char) {
if !self.current_char.is_whitespace() {
if self.current_char.is_ascii_digit() {
self.scan_number()?;
} else if let Some(identifier_end) =
@ -575,9 +575,12 @@ impl<'a> TokenizerState<'a> {
) -> Result<(), TokenizerError> {
self.advance(1)?;
let value = self.extract_value()?[2..].to_string();
match u64::from_str_radix(&value, radix) {
Ok(_) => self.add(radix_token_type, Some(value)),
Err(_) => self.add(self.token_types.identifier, None),
// Validate if the string consists only of valid hex digits
if value.chars().all(|c| c.is_digit(radix)) {
self.add(radix_token_type, Some(value))
} else {
self.add(self.token_types.identifier, None)
}
}

2
tests/dialects/test_bigquery.py
View file

@ -865,6 +865,7 @@ LANGUAGE js AS
"presto": "SHA256(x)",
"trino": "SHA256(x)",
"postgres": "SHA256(x)",
"duckdb": "SHA256(x)",
},
write={
"bigquery": "SHA256(x)",
@ -875,6 +876,7 @@ LANGUAGE js AS
"redshift": "SHA2(x, 256)",
"trino": "SHA256(x)",
"duckdb": "SHA256(x)",
"snowflake": "SHA2(x, 256)",
},
)
self.validate_all(

71
tests/dialects/test_clickhouse.py
View file

@ -29,6 +29,10 @@ class TestClickhouse(Validator):
self.assertEqual(expr.sql(dialect="clickhouse"), "COUNT(x)")
self.assertIsNone(expr._meta)
self.validate_identity("SELECT 1 OR (1 = 2)")
self.validate_identity("SELECT 1 AND (1 = 2)")
self.validate_identity("SELECT json.a.:Int64")
self.validate_identity("SELECT json.a.:JSON.b.:Int64")
self.validate_identity("WITH arrayJoin([(1, [2, 3])]) AS arr SELECT arr")
self.validate_identity("CAST(1 AS Bool)")
self.validate_identity("SELECT toString(CHAR(104.1, 101, 108.9, 108.9, 111, 32))")
@ -85,6 +89,7 @@ class TestClickhouse(Validator):
self.validate_identity("SELECT exponentialTimeDecayedAvg(60)(a, b)")
self.validate_identity("levenshteinDistance(col1, col2)", "editDistance(col1, col2)")
self.validate_identity("SELECT * FROM foo WHERE x GLOBAL IN (SELECT * FROM bar)")
self.validate_identity("SELECT * FROM foo WHERE x GLOBAL NOT IN (SELECT * FROM bar)")
self.validate_identity("POSITION(haystack, needle)")
self.validate_identity("POSITION(haystack, needle, position)")
self.validate_identity("CAST(x AS DATETIME)", "CAST(x AS DateTime)")
@ -158,6 +163,21 @@ class TestClickhouse(Validator):
self.validate_identity(
"CREATE TABLE t (foo String CODEC(LZ4HC(9), ZSTD, DELTA), size String ALIAS formatReadableSize(size_bytes), INDEX idx1 a TYPE bloom_filter(0.001) GRANULARITY 1, INDEX idx2 a TYPE set(100) GRANULARITY 2, INDEX idx3 a TYPE minmax GRANULARITY 3)"
)
self.validate_identity(
"SELECT generate_series FROM generate_series(0, 10) AS g(x)",
)
self.validate_identity(
"SELECT and(1, 2)",
"SELECT 1 AND 2",
)
self.validate_identity(
"SELECT or(1, 2)",
"SELECT 1 OR 2",
)
self.validate_identity(
"SELECT generate_series FROM generate_series(0, 10) AS g",
"SELECT generate_series FROM generate_series(0, 10) AS g(generate_series)",
)
self.validate_identity(
"INSERT INTO tab VALUES ({'key1': 1, 'key2': 10}), ({'key1': 2, 'key2': 20}), ({'key1': 3, 'key2': 30})",
"INSERT INTO tab VALUES (map('key1', 1, 'key2', 10)), (map('key1', 2, 'key2', 20)), (map('key1', 3, 'key2', 30))",
@ -179,6 +199,13 @@ class TestClickhouse(Validator):
"SELECT SUM(1) AS impressions FROM (SELECT ['Istanbul', 'Berlin', 'Bobruisk'] AS cities) WHERE arrayJoin(cities) IN ('Istanbul', 'Berlin')",
)
self.validate_all(
"SELECT CAST(STR_TO_DATE(SUBSTRING(a.eta, 1, 10), '%Y-%m-%d') AS Nullable(DATE))",
read={
"clickhouse": "SELECT CAST(STR_TO_DATE(SUBSTRING(a.eta, 1, 10), '%Y-%m-%d') AS Nullable(DATE))",
"oracle": "SELECT to_date(substr(a.eta, 1,10), 'YYYY-MM-DD')",
},
)
self.validate_all(
"CHAR(67) || CHAR(65) || CHAR(84)",
read={
@ -201,13 +228,13 @@ class TestClickhouse(Validator):
},
)
self.validate_all(
"SELECT CAST(STR_TO_DATE('05 12 2000', '%d %m %Y') AS DATE)",
"SELECT CAST(STR_TO_DATE('05 12 2000', '%d %m %Y') AS Nullable(DATE))",
read={
"clickhouse": "SELECT CAST(STR_TO_DATE('05 12 2000', '%d %m %Y') AS DATE)",
"clickhouse": "SELECT CAST(STR_TO_DATE('05 12 2000', '%d %m %Y') AS Nullable(DATE))",
"postgres": "SELECT TO_DATE('05 12 2000', 'DD MM YYYY')",
},
write={
"clickhouse": "SELECT CAST(STR_TO_DATE('05 12 2000', '%d %m %Y') AS DATE)",
"clickhouse": "SELECT CAST(STR_TO_DATE('05 12 2000', '%d %m %Y') AS Nullable(DATE))",
"postgres": "SELECT CAST(CAST(TO_DATE('05 12 2000', 'DD MM YYYY') AS TIMESTAMP) AS DATE)",
},
)
@ -226,9 +253,9 @@ class TestClickhouse(Validator):
},
)
self.validate_all(
"SELECT a, b FROM (SELECT * FROM x) AS t",
"SELECT a, b FROM (SELECT * FROM x) AS t(a, b)",
read={
"clickhouse": "SELECT a, b FROM (SELECT * FROM x) AS t",
"clickhouse": "SELECT a, b FROM (SELECT * FROM x) AS t(a, b)",
"duckdb": "SELECT a, b FROM (SELECT * FROM x) AS t(a, b)",
},
)
@ -557,6 +584,7 @@ class TestClickhouse(Validator):
self.validate_identity(
"SELECT COUNT(1) FROM table SETTINGS additional_table_filters = {'a': 'b', 'c': 'd'}"
)
self.validate_identity("SELECT arrayConcat([1, 2], [3, 4])")
def test_clickhouse_values(self):
values = exp.select("*").from_(
@ -682,6 +710,33 @@ class TestClickhouse(Validator):
with self.subTest(f"Casting to ClickHouse {data_type}"):
self.validate_identity(f"SELECT CAST(val AS {data_type})")
def test_aggregate_function_column_with_any_keyword(self):
# Regression test for https://github.com/tobymao/sqlglot/issues/4723
self.validate_all(
"""
CREATE TABLE my_db.my_table
(
someId UUID,
aggregatedColumn AggregateFunction(any, String),
aggregatedColumnWithParams AggregateFunction(any(somecolumn), String),
)
ENGINE = AggregatingMergeTree()
ORDER BY (someId)
""",
write={
"clickhouse": """CREATE TABLE my_db.my_table (
someId UUID,
aggregatedColumn AggregateFunction(any, String),
aggregatedColumnWithParams AggregateFunction(any(somecolumn), String)
)
ENGINE=AggregatingMergeTree()
ORDER BY (
someId
)""",
},
pretty=True,
)
def test_ddl(self):
db_table_expr = exp.Table(this=None, db=exp.to_identifier("foo"), catalog=None)
create_with_cluster = exp.Create(
@ -1061,13 +1116,15 @@ LIFETIME(MIN 0 MAX 0)""",
CREATE TABLE t (
a AggregateFunction(quantiles(0.5, 0.9), UInt64),
b AggregateFunction(quantiles, UInt64),
c SimpleAggregateFunction(sum, Float64)
c SimpleAggregateFunction(sum, Float64),
d AggregateFunction(count)
)""",
write={
"clickhouse": """CREATE TABLE t (
a AggregateFunction(quantiles(0.5, 0.9), UInt64),
b AggregateFunction(quantiles, UInt64),
c SimpleAggregateFunction(sum, Float64)
c SimpleAggregateFunction(sum, Float64),
d AggregateFunction(count)
)"""
},
pretty=True,

37
tests/dialects/test_dialect.py
View file

@ -168,6 +168,18 @@ class TestDialect(Validator):
self.assertFalse(snowflake_class in {"bigquery", "redshift"})
self.assertFalse(snowflake_object in {"bigquery", "redshift"})
def test_compare_dialect_versions(self):
ddb_v1 = Dialect.get_or_raise("duckdb, version=1.0")
ddb_v1_2 = Dialect.get_or_raise("duckdb, foo=bar, version=1.0")
ddb_v2 = Dialect.get_or_raise("duckdb, version=2.2.4")
ddb_latest = Dialect.get_or_raise("duckdb")
self.assertTrue(ddb_latest.version > ddb_v2.version)
self.assertTrue(ddb_v1.version < ddb_v2.version)
self.assertTrue(ddb_v1.version == ddb_v1_2.version)
self.assertTrue(ddb_latest.version == Dialect.get_or_raise("duckdb").version)
def test_cast(self):
self.validate_all(
"CAST(a AS TEXT)",
@ -3015,7 +3027,7 @@ FROM subquery2""",
"databricks": "SELECT * FROM EXPLODE(SEQUENCE(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), INTERVAL '1' WEEK))",
"duckdb": "SELECT * FROM UNNEST(CAST(GENERATE_SERIES(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), (7 * INTERVAL '1' DAY)) AS DATE[]))",
"mysql": "WITH RECURSIVE _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATE_ADD(date_value, INTERVAL 1 WEEK) AS DATE) FROM _generated_dates WHERE CAST(DATE_ADD(date_value, INTERVAL 1 WEEK) AS DATE) <= CAST('2020-02-01' AS DATE)) SELECT * FROM (SELECT date_value FROM _generated_dates) AS _generated_dates",
"postgres": "SELECT * FROM (SELECT CAST(value AS DATE) FROM GENERATE_SERIES(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), INTERVAL '1 WEEK') AS value) AS _unnested_generate_series",
"postgres": "SELECT * FROM (SELECT CAST(value AS DATE) FROM GENERATE_SERIES(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), INTERVAL '1 WEEK') AS _t(value)) AS _unnested_generate_series",
"presto": "SELECT * FROM UNNEST(SEQUENCE(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), (1 * INTERVAL '7' DAY)))",
"redshift": "WITH RECURSIVE _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_value) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_value) AS DATE) <= CAST('2020-02-01' AS DATE)) SELECT * FROM (SELECT date_value FROM _generated_dates) AS _generated_dates",
"snowflake": "SELECT * FROM (SELECT DATEADD(WEEK, CAST(value AS INT), CAST('2020-01-01' AS DATE)) AS value FROM TABLE(FLATTEN(INPUT => ARRAY_GENERATE_RANGE(0, (DATEDIFF(WEEK, CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE)) + 1 - 1) + 1))) AS _u(seq, key, path, index, value, this))",
@ -3335,3 +3347,26 @@ FROM subquery2""",
"tsql": "SCHEMA_NAME()",
},
)
def test_integer_hex_strings(self):
# Hex strings such as 0xCC represent INTEGER values in the read dialects
integer_dialects = ("bigquery", "clickhouse")
for read_dialect in integer_dialects:
for write_dialect in (
"",
"duckdb",
"databricks",
"snowflake",
"spark",
"redshift",
):
with self.subTest(f"Testing hex string -> INTEGER evaluation for {read_dialect}"):
self.assertEqual(
parse_one("SELECT 0xCC", read=read_dialect).sql(write_dialect), "SELECT 204"
)
for other_integer_dialects in integer_dialects:
self.assertEqual(
parse_one("SELECT 0xCC", read=read_dialect).sql(other_integer_dialects),
"SELECT 0xCC",
)

29
tests/dialects/test_duckdb.py
View file

@ -314,6 +314,10 @@ class TestDuckDB(Validator):
self.validate_identity(
"""SELECT '{ "family": "anatidae", "species": [ "duck", "goose", "swan", null ] }' ->> ['$.family', '$.species']""",
)
self.validate_identity(
"SELECT 20_000 AS literal",
"SELECT 20000 AS literal",
)
self.validate_identity(
"""SELECT JSON_EXTRACT_STRING('{ "family": "anatidae", "species": [ "duck", "goose", "swan", null ] }', ['$.family', '$.species'])""",
"""SELECT '{ "family": "anatidae", "species": [ "duck", "goose", "swan", null ] }' ->> ['$.family', '$.species']""",
@ -403,6 +407,14 @@ class TestDuckDB(Validator):
self.validate_all("0x1010", write={"": "0 AS x1010"})
self.validate_all("x ~ y", write={"duckdb": "REGEXP_MATCHES(x, y)"})
self.validate_all("SELECT * FROM 'x.y'", write={"duckdb": 'SELECT * FROM "x.y"'})
self.validate_all(
"COUNT_IF(x)",
write={
"duckdb": "COUNT_IF(x)",
"duckdb, version=1.0": "SUM(CASE WHEN x THEN 1 ELSE 0 END)",
"duckdb, version=1.2": "COUNT_IF(x)",
},
)
self.validate_all(
"SELECT STRFTIME(CAST('2020-01-01' AS TIMESTAMP), CONCAT('%Y', '%m'))",
write={
@ -410,6 +422,13 @@ class TestDuckDB(Validator):
"tsql": "SELECT FORMAT(CAST('2020-01-01' AS DATETIME2), CONCAT('yyyy', 'MM'))",
},
)
self.validate_all(
"""SELECT CAST('{"x": 1}' AS JSON)""",
read={
"duckdb": """SELECT '{"x": 1}'::JSON""",
"postgres": """SELECT '{"x": 1}'::JSONB""",
},
)
self.validate_all(
"SELECT * FROM produce PIVOT(SUM(sales) FOR quarter IN ('Q1', 'Q2'))",
read={
@ -910,6 +929,10 @@ class TestDuckDB(Validator):
"postgres": "SELECT 'ThOmAs' ~* 'thomas'",
},
)
self.validate_identity(
"SELECT DATE_ADD(CAST('2020-01-01' AS DATE), INTERVAL 1 DAY)",
"SELECT CAST('2020-01-01' AS DATE) + INTERVAL '1' DAY",
)
def test_array_index(self):
with self.assertLogs(helper_logger) as cm:
@ -929,15 +952,15 @@ class TestDuckDB(Validator):
)
self.validate_identity(
"""SELECT LIST_VALUE(1)[i]""",
"""SELECT ([1])[i]""",
"""SELECT [1][i]""",
)
self.validate_identity(
"""{'x': LIST_VALUE(1)[i]}""",
"""{'x': ([1])[i]}""",
"""{'x': [1][i]}""",
)
self.validate_identity(
"""SELECT LIST_APPLY(RANGE(1, 4), i -> {'f1': LIST_VALUE(1, 2, 3)[i], 'f2': LIST_VALUE(1, 2, 3)[i]})""",
"""SELECT LIST_APPLY(RANGE(1, 4), i -> {'f1': ([1, 2, 3])[i], 'f2': ([1, 2, 3])[i]})""",
"""SELECT LIST_APPLY(RANGE(1, 4), i -> {'f1': [1, 2, 3][i], 'f2': [1, 2, 3][i]})""",
)
self.assertEqual(

27
tests/dialects/test_dune.py
View file

@ -1,3 +1,4 @@
from sqlglot import exp
from tests.dialects.test_dialect import Validator
@ -8,16 +9,28 @@ class TestDune(Validator):
self.validate_identity("CAST(x AS INT256)")
self.validate_identity("CAST(x AS UINT256)")
for hex_literal in (
"deadbeef",
"deadbeefdead",
"deadbeefdeadbeef",
"deadbeefdeadbeefde",
"deadbeefdeadbeefdead",
"deadbeefdeadbeefdeadbeef",
"deadbeefdeadbeefdeadbeefdeadbeef",
):
with self.subTest(f"Transpiling hex literal {hex_literal}"):
self.parse_one(f"0x{hex_literal}").assert_is(exp.HexString)
self.validate_all(
"SELECT 0xdeadbeef",
f"SELECT 0x{hex_literal}",
read={
"dune": "SELECT X'deadbeef'",
"postgres": "SELECT x'deadbeef'",
"trino": "SELECT X'deadbeef'",
"dune": f"SELECT X'{hex_literal}'",
"postgres": f"SELECT x'{hex_literal}'",
"trino": f"SELECT X'{hex_literal}'",
},
write={
"dune": "SELECT 0xdeadbeef",
"postgres": "SELECT x'deadbeef'",
"trino": "SELECT X'deadbeef'",
"dune": f"SELECT 0x{hex_literal}",
"postgres": f"SELECT x'{hex_literal}'",
"trino": f"SELECT x'{hex_literal}'",
},
)

32
tests/dialects/test_mysql.py
View file

@ -14,6 +14,7 @@ class TestMySQL(Validator):
self.validate_identity(f"CREATE TABLE t (id {t} UNSIGNED)")
self.validate_identity(f"CREATE TABLE t (id {t}(10) UNSIGNED)")
self.validate_identity("CREATE TABLE bar (abacate DOUBLE(10, 2) UNSIGNED)")
self.validate_identity("CREATE TABLE t (id DECIMAL(20, 4) UNSIGNED)")
self.validate_identity("CREATE TABLE foo (a BIGINT, UNIQUE (b) USING BTREE)")
self.validate_identity("CREATE TABLE foo (id BIGINT)")
@ -149,6 +150,10 @@ class TestMySQL(Validator):
"sqlite": "CREATE TABLE x (id INTEGER NOT NULL AUTOINCREMENT PRIMARY KEY)",
},
)
self.validate_identity("ALTER TABLE t ALTER INDEX i INVISIBLE")
self.validate_identity("ALTER TABLE t ALTER INDEX i VISIBLE")
self.validate_identity("ALTER TABLE t ALTER COLUMN c SET INVISIBLE")
self.validate_identity("ALTER TABLE t ALTER COLUMN c SET VISIBLE")
def test_identity(self):
self.validate_identity("SELECT HIGH_PRIORITY STRAIGHT_JOIN SQL_CALC_FOUND_ROWS * FROM t")
@ -385,16 +390,16 @@ class TestMySQL(Validator):
def test_hexadecimal_literal(self):
write_CC = {
"bigquery": "SELECT 0xCC",
"clickhouse": "SELECT 0xCC",
"bigquery": "SELECT FROM_HEX('CC')",
"clickhouse": UnsupportedError,
"databricks": "SELECT X'CC'",
"drill": "SELECT 204",
"duckdb": "SELECT 204",
"duckdb": "SELECT FROM_HEX('CC')",
"hive": "SELECT 204",
"mysql": "SELECT x'CC'",
"oracle": "SELECT 204",
"postgres": "SELECT x'CC'",
"presto": "SELECT 204",
"presto": "SELECT x'CC'",
"redshift": "SELECT 204",
"snowflake": "SELECT x'CC'",
"spark": "SELECT X'CC'",
@ -402,20 +407,20 @@ class TestMySQL(Validator):
"starrocks": "SELECT x'CC'",
"tableau": "SELECT 204",
"teradata": "SELECT X'CC'",
"trino": "SELECT X'CC'",
"trino": "SELECT x'CC'",
"tsql": "SELECT 0xCC",
}
write_CC_with_leading_zeros = {
"bigquery": "SELECT 0x0000CC",
"clickhouse": "SELECT 0x0000CC",
"bigquery": "SELECT FROM_HEX('0000CC')",
"clickhouse": UnsupportedError,
"databricks": "SELECT X'0000CC'",
"drill": "SELECT 204",
"duckdb": "SELECT 204",
"duckdb": "SELECT FROM_HEX('0000CC')",
"hive": "SELECT 204",
"mysql": "SELECT x'0000CC'",
"oracle": "SELECT 204",
"postgres": "SELECT x'0000CC'",
"presto": "SELECT 204",
"presto": "SELECT x'0000CC'",
"redshift": "SELECT 204",
"snowflake": "SELECT x'0000CC'",
"spark": "SELECT X'0000CC'",
@ -423,7 +428,7 @@ class TestMySQL(Validator):
"starrocks": "SELECT x'0000CC'",
"tableau": "SELECT 204",
"teradata": "SELECT X'0000CC'",
"trino": "SELECT X'0000CC'",
"trino": "SELECT x'0000CC'",
"tsql": "SELECT 0x0000CC",
}
@ -728,6 +733,13 @@ class TestMySQL(Validator):
},
)
self.validate_all(
"CURDATE()",
write={
"mysql": "CURRENT_DATE",
"postgres": "CURRENT_DATE",
},
)
self.validate_all(
"SELECT CONCAT('11', '22')",
read={

7
tests/dialects/test_oracle.py
View file

@ -117,6 +117,13 @@ class TestOracle(Validator):
"SELECT * FROM t START WITH col CONNECT BY NOCYCLE PRIOR col1 = col2"
)
self.validate_all(
"SELECT TRIM('|' FROM '||Hello ||| world||')",
write={
"clickhouse": "SELECT TRIM(BOTH '|' FROM '||Hello ||| world||')",
"oracle": "SELECT TRIM('|' FROM '||Hello ||| world||')",
},
)
self.validate_all(
"SELECT department_id, department_name INTO v_department_id, v_department_name FROM departments FETCH FIRST 1 ROWS ONLY",
write={

10
tests/dialects/test_postgres.py
View file

@ -1358,3 +1358,13 @@ CROSS JOIN JSON_ARRAY_ELEMENTS(CAST(JSON_EXTRACT_PATH(tbox, 'boxes') AS JSON)) A
self.validate_identity("ANALYZE TBL(col1, col2)")
self.validate_identity("ANALYZE VERBOSE SKIP_LOCKED TBL(col1, col2)")
self.validate_identity("ANALYZE BUFFER_USAGE_LIMIT 1337 TBL")
def test_recursive_cte(self):
for kind in ("BREADTH", "DEPTH"):
self.validate_identity(
f"WITH RECURSIVE search_tree(id, link, data) AS (SELECT t.id, t.link, t.data FROM tree AS t UNION ALL SELECT t.id, t.link, t.data FROM tree AS t, search_tree AS st WHERE t.id = st.link) SEARCH {kind} FIRST BY id SET ordercol SELECT * FROM search_tree ORDER BY ordercol"
)
self.validate_identity(
"WITH RECURSIVE search_graph(id, link, data, depth) AS (SELECT g.id, g.link, g.data, 1 FROM graph AS g UNION ALL SELECT g.id, g.link, g.data, sg.depth + 1 FROM graph AS g, search_graph AS sg WHERE g.id = sg.link) CYCLE id SET is_cycle USING path SELECT * FROM search_graph"
)

59
tests/dialects/test_snowflake.py
View file

@ -1427,18 +1427,10 @@ class TestSnowflake(Validator):
"CREATE TABLE t (id INT TAG (key1='value_1', key2='value_2'))",
)
self.validate_identity("USE SECONDARY ROLES ALL")
self.validate_identity("USE SECONDARY ROLES NONE")
self.validate_identity("USE SECONDARY ROLES a, b, c")
self.validate_identity("CREATE SECURE VIEW table1 AS (SELECT a FROM table2)")
self.validate_identity(
"""create external table et2(
col1 date as (parse_json(metadata$external_table_partition):COL1::date),
col2 varchar as (parse_json(metadata$external_table_partition):COL2::varchar),
col3 number as (parse_json(metadata$external_table_partition):COL3::number))
partition by (col1,col2,col3)
location=@s2/logs/
partition_type = user_specified
file_format = (type = parquet)""",
"CREATE EXTERNAL TABLE et2 (col1 DATE AS (CAST(GET_PATH(PARSE_JSON(metadata$external_table_partition), 'COL1') AS DATE)), col2 VARCHAR AS (CAST(GET_PATH(PARSE_JSON(metadata$external_table_partition), 'COL2') AS VARCHAR)), col3 DECIMAL(38, 0) AS (CAST(GET_PATH(PARSE_JSON(metadata$external_table_partition), 'COL3') AS DECIMAL(38, 0)))) LOCATION @s2/logs/ PARTITION BY (col1, col2, col3) partition_type=user_specified file_format=(type = parquet)",
)
self.validate_identity("CREATE OR REPLACE VIEW foo (uid) COPY GRANTS AS (SELECT 1)")
self.validate_identity("CREATE TABLE geospatial_table (id INT, g GEOGRAPHY)")
self.validate_identity("CREATE MATERIALIZED VIEW a COMMENT='...' AS SELECT 1 FROM x")
@ -1504,6 +1496,17 @@ class TestSnowflake(Validator):
"CREATE SEQUENCE seq1 WITH START=1 INCREMENT=1 ORDER",
"CREATE SEQUENCE seq1 START=1 INCREMENT=1 ORDER",
)
self.validate_identity(
"""create external table et2(
col1 date as (parse_json(metadata$external_table_partition):COL1::date),
col2 varchar as (parse_json(metadata$external_table_partition):COL2::varchar),
col3 number as (parse_json(metadata$external_table_partition):COL3::number))
partition by (col1,col2,col3)
location=@s2/logs/
partition_type = user_specified
file_format = (type = parquet)""",
"CREATE EXTERNAL TABLE et2 (col1 DATE AS (CAST(GET_PATH(PARSE_JSON(metadata$external_table_partition), 'COL1') AS DATE)), col2 VARCHAR AS (CAST(GET_PATH(PARSE_JSON(metadata$external_table_partition), 'COL2') AS VARCHAR)), col3 DECIMAL(38, 0) AS (CAST(GET_PATH(PARSE_JSON(metadata$external_table_partition), 'COL3') AS DECIMAL(38, 0)))) LOCATION @s2/logs/ PARTITION BY (col1, col2, col3) partition_type=user_specified file_format=(type = parquet)",
)
self.validate_all(
"CREATE TABLE orders_clone CLONE orders",
@ -2062,6 +2065,40 @@ MATCH_RECOGNIZE (
self.validate_identity("SHOW TERSE USERS")
self.validate_identity("SHOW USERS LIKE '_foo%' STARTS WITH 'bar' LIMIT 5 FROM 'baz'")
def test_show_databases(self):
self.validate_identity("SHOW TERSE DATABASES")
self.validate_identity(
"SHOW TERSE DATABASES HISTORY LIKE 'foo' STARTS WITH 'bla' LIMIT 5 FROM 'bob' WITH PRIVILEGES USAGE, MODIFY"
)
ast = parse_one("SHOW DATABASES IN ACCOUNT", read="snowflake")
self.assertEqual(ast.this, "DATABASES")
self.assertEqual(ast.args.get("scope_kind"), "ACCOUNT")
def test_show_functions(self):
self.validate_identity("SHOW FUNCTIONS")
self.validate_identity("SHOW FUNCTIONS LIKE 'foo' IN CLASS bla")
ast = parse_one("SHOW FUNCTIONS IN ACCOUNT", read="snowflake")
self.assertEqual(ast.this, "FUNCTIONS")
self.assertEqual(ast.args.get("scope_kind"), "ACCOUNT")
def test_show_procedures(self):
self.validate_identity("SHOW PROCEDURES")
self.validate_identity("SHOW PROCEDURES LIKE 'foo' IN APPLICATION app")
self.validate_identity("SHOW PROCEDURES LIKE 'foo' IN APPLICATION PACKAGE pkg")
ast = parse_one("SHOW PROCEDURES IN ACCOUNT", read="snowflake")
self.assertEqual(ast.this, "PROCEDURES")
self.assertEqual(ast.args.get("scope_kind"), "ACCOUNT")
def test_show_warehouses(self):
self.validate_identity("SHOW WAREHOUSES")
self.validate_identity("SHOW WAREHOUSES LIKE 'foo' WITH PRIVILEGES USAGE, MODIFY")
ast = parse_one("SHOW WAREHOUSES", read="snowflake")
self.assertEqual(ast.this, "WAREHOUSES")
def test_show_schemas(self):
self.validate_identity(
"show terse schemas in database db1 starts with 'a' limit 10 from 'b'",

6
tests/dialects/test_spark.py
View file

@ -316,7 +316,8 @@ TBLPROPERTIES (
write={
"databricks": "SELECT TRY_ELEMENT_AT(ARRAY(1, 2, 3), 2)",
"spark": "SELECT TRY_ELEMENT_AT(ARRAY(1, 2, 3), 2)",
"duckdb": "SELECT ([1, 2, 3])[2]",
"duckdb": "SELECT [1, 2, 3][2]",
"duckdb, version=1.1.0": "SELECT ([1, 2, 3])[2]",
"presto": "SELECT ELEMENT_AT(ARRAY[1, 2, 3], 2)",
},
)
@ -356,7 +357,8 @@ TBLPROPERTIES (
},
write={
"databricks": "SELECT TRY_ELEMENT_AT(MAP(1, 'a', 2, 'b'), 2)",
"duckdb": "SELECT (MAP([1, 2], ['a', 'b'])[2])[1]",
"duckdb": "SELECT MAP([1, 2], ['a', 'b'])[2]",
"duckdb, version=1.1.0": "SELECT (MAP([1, 2], ['a', 'b'])[2])[1]",
"spark": "SELECT TRY_ELEMENT_AT(MAP(1, 'a', 2, 'b'), 2)",
},
)

1
tests/dialects/test_sqlite.py
View file

@ -108,6 +108,7 @@ class TestSQLite(Validator):
"SELECT * FROM station WHERE city IS NOT ''",
"SELECT * FROM station WHERE NOT city IS ''",
)
self.validate_identity("SELECT JSON_OBJECT('col1', 1, 'col2', '1')")
def test_strftime(self):
self.validate_identity("SELECT STRFTIME('%Y/%m/%d', 'now')")

1
tests/dialects/test_trino.py
View file

@ -5,6 +5,7 @@ class TestTrino(Validator):
dialect = "trino"
def test_trino(self):
self.validate_identity("JSON_QUERY(m.properties, 'lax $.area' OMIT QUOTES NULL ON ERROR)")
self.validate_identity("JSON_EXTRACT(content, json_path)")
self.validate_identity("JSON_QUERY(content, 'lax $.HY.*')")
self.validate_identity("JSON_QUERY(content, 'strict $.HY.*' WITH WRAPPER)")

2
tests/dialects/test_tsql.py
View file

@ -1705,6 +1705,8 @@ WHERE
"spark": "SELECT * FROM A LIMIT 3",
},
)
self.validate_identity("SELECT TOP 10 PERCENT")
self.validate_identity("SELECT TOP 10 PERCENT WITH TIES")
def test_format(self):
self.validate_identity("SELECT FORMAT(foo, 'dddd', 'de-CH')")

7
tests/fixtures/optimizer/annotate_functions.sql vendored
View file

@ -315,3 +315,10 @@ STRING;
# dialect: bigquery
STRING(timestamp_expr, timezone);
STRING;
--------------------------------------
-- Snowflake
--------------------------------------
LEAST(x::DECIMAL(18, 2));
DECIMAL(18, 2);

18
tests/fixtures/optimizer/qualify_columns.sql vendored
View file

@ -259,6 +259,20 @@ WITH T1 AS (SELECT 1 AS C1, 1 AS C2, 'Y' AS TOP_PARENT_INDICATOR, 1 AS ID FROM D
SELECT * FROM ROWS FROM (GENERATE_SERIES(1, 3), GENERATE_SERIES(10, 12)) AS t(a, b);
SELECT t.a AS a, t.b AS b FROM ROWS FROM (GENERATE_SERIES(1, 3), GENERATE_SERIES(10, 12)) AS t(a, b);
# execute: false
# dialect: clickhouse
SELECT generate_series FROM generate_series(0, 10) AS g;
SELECT g.generate_series AS generate_series FROM generate_series(0, 10) AS g(generate_series);
# execute: false
# dialect: snowflake
SELECT * FROM quarterly_sales PIVOT(SUM(amount) FOR quarter IN (ANY ORDER BY quarter)) ORDER BY empid;
SELECT * FROM QUARTERLY_SALES AS QUARTERLY_SALES PIVOT(SUM(QUARTERLY_SALES.AMOUNT) FOR QUARTERLY_SALES.QUARTER IN (ANY ORDER BY QUARTER)) AS _Q_0 ORDER BY _Q_0.EMPID;
# execute: false
SELECT PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY x) AS x FROM t;
SELECT PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY t.x) AS x FROM t AS t;
--------------------------------------
-- Derived tables
--------------------------------------
@ -752,6 +766,10 @@ WITH RECURSIVE t(c) AS (SELECT 1 AS c UNION ALL SELECT t.c + 1 AS c FROM t AS t
WITH RECURSIVE t AS (SELECT 1 AS c UNION ALL SELECT c + 1 AS c FROM t WHERE c <= 10) SELECT c FROM t;
WITH RECURSIVE t AS (SELECT 1 AS c UNION ALL SELECT t.c + 1 AS c FROM t AS t WHERE t.c <= 10) SELECT t.c AS c FROM t AS t;
# title: expand DISTINCT ON ordinals / projection names
SELECT DISTINCT ON (new_col, b + 1, 1) t1.a AS new_col FROM x AS t1 ORDER BY new_col;
SELECT DISTINCT ON (new_col, t1.b + 1, new_col) t1.a AS new_col FROM x AS t1 ORDER BY new_col;
--------------------------------------
-- Wrapped tables / join constructs
--------------------------------------

19
tests/test_diff.py
View file

@ -240,9 +240,26 @@ class TestDiff(unittest.TestCase):
],
)
with self.assertRaises(ValueError):
self._validate_delta_only(
diff_delta_only(
expr_src, expr_tgt, matchings=[(expr_src, expr_tgt), (expr_src, expr_tgt)]
),
[
Insert(expression=exp.Literal.number(2)),
Insert(expression=exp.Literal.number(3)),
Insert(expression=exp.Literal.number(4)),
],
)
expr_tgt.selects[0].replace(expr_src.selects[0])
self._validate_delta_only(
diff_delta_only(expr_src, expr_tgt, matchings=[(expr_src, expr_tgt)]),
[
Insert(expression=exp.Literal.number(2)),
Insert(expression=exp.Literal.number(3)),
Insert(expression=exp.Literal.number(4)),
],
)
def test_identifier(self):

22
tests/test_optimizer.py
View file

@ -216,6 +216,28 @@ class TestOptimizer(unittest.TestCase):
)
def test_qualify_tables(self):
self.assertEqual(
optimizer.qualify_tables.qualify_tables(
parse_one(
"WITH cte AS (SELECT * FROM t) SELECT * FROM cte PIVOT(SUM(c) FOR v IN ('x', 'y'))"
),
db="db",
catalog="catalog",
).sql(),
"WITH cte AS (SELECT * FROM catalog.db.t AS t) SELECT * FROM cte AS cte PIVOT(SUM(c) FOR v IN ('x', 'y')) AS _q_0",
)
self.assertEqual(
optimizer.qualify_tables.qualify_tables(
parse_one(
"WITH cte AS (SELECT * FROM t) SELECT * FROM cte PIVOT(SUM(c) FOR v IN ('x', 'y')) AS pivot_alias"
),
db="db",
catalog="catalog",
).sql(),
"WITH cte AS (SELECT * FROM catalog.db.t AS t) SELECT * FROM cte AS cte PIVOT(SUM(c) FOR v IN ('x', 'y')) AS pivot_alias",
)
self.assertEqual(
optimizer.qualify_tables.qualify_tables(
parse_one("select a from b"), catalog="catalog"

20
tests/test_transforms.py
View file

@ -1,6 +1,6 @@
import unittest
from sqlglot import parse_one
from sqlglot import parse_one, expressions as exp
from sqlglot.transforms import (
eliminate_distinct_on,
eliminate_join_marks,
@ -147,7 +147,7 @@ class TestTransforms(unittest.TestCase):
self.validate(
eliminate_qualify,
"SELECT x AS z FROM y QUALIFY ROW_NUMBER() OVER (PARTITION BY z)",
"SELECT z FROM (SELECT x AS z, ROW_NUMBER() OVER (PARTITION BY x) AS _w, x FROM y) AS _t WHERE _w",
"SELECT z FROM (SELECT x AS z, ROW_NUMBER() OVER (PARTITION BY x) AS _w FROM y) AS _t WHERE _w",
)
self.validate(
eliminate_qualify,
@ -162,7 +162,12 @@ class TestTransforms(unittest.TestCase):
self.validate(
eliminate_qualify,
"SELECT y.x AS x, y.t AS z FROM y QUALIFY ROW_NUMBER() OVER (PARTITION BY x ORDER BY x DESC, z)",
"SELECT x, z FROM (SELECT y.x AS x, y.t AS z, ROW_NUMBER() OVER (PARTITION BY y.x ORDER BY y.x DESC, y.t) AS _w, y.t FROM y) AS _t WHERE _w",
"SELECT x, z FROM (SELECT y.x AS x, y.t AS z, ROW_NUMBER() OVER (PARTITION BY y.x ORDER BY y.x DESC, y.t) AS _w FROM y) AS _t WHERE _w",
)
self.validate(
eliminate_qualify,
"select max(col) over (partition by col_id) as col, from some_table qualify row_number() over (partition by col_id order by col asc)=1",
"SELECT col FROM (SELECT MAX(col) OVER (PARTITION BY col_id) AS col, ROW_NUMBER() OVER (PARTITION BY col_id ORDER BY MAX(col) OVER (PARTITION BY col_id) ASC) AS _w, col_id FROM some_table) AS _t WHERE _w = 1",
)
def test_remove_precision_parameterized_types(self):
@ -251,3 +256,12 @@ class TestTransforms(unittest.TestCase):
f"SELECT table1.id, table2.cloumn1, table3.id FROM table1 LEFT JOIN table2 ON table1.id = table2.id LEFT JOIN (SELECT tableInner1.id FROM tableInner1 LEFT JOIN tableInner2 ON tableInner1.id = tableInner2.id) {alias}table3 ON table1.id = table3.id",
dialect,
)
# if multiple conditions, we check that after transformations the tree remains consistent
s = "select a.id from a, b where a.id = b.id (+) AND b.d (+) = const"
tree = eliminate_join_marks(parse_one(s, dialect=dialect))
assert all(type(t.parent_select) is exp.Select for t in tree.find_all(exp.Table))
assert (
tree.sql(dialect=dialect)
== "SELECT a.id FROM a LEFT JOIN b ON a.id = b.id AND b.d = const"
)

5
tests/test_transpile.py
View file

@ -23,6 +23,11 @@ class TestTranspile(unittest.TestCase):
def test_weird_chars(self):
self.assertEqual(transpile("0Êß")[0], "0 AS Êß")
self.assertEqual(
# Ideographic space after SELECT (\u3000)
transpile("SELECT　* FROM t WHERE c = 1")[0],
"SELECT * FROM t WHERE c = 1",
)
def test_alias(self):
self.assertEqual(transpile("SELECT SUM(y) KEEP")[0], "SELECT SUM(y) AS KEEP")