dbcli/sqlglot
1
0
Fork 0
Code Issues Activity

Merging upstream version 26.14.0.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-04-16 09:04:43 +02:00
parent 68f1150572
commit e9f53ab285
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
84 changed files with 63872 additions and 61909 deletions
Download patch file Download diff file Expand all files Collapse all files

83
CHANGELOG.md
View file

@ -1,6 +1,86 @@
Changelog Changelog
========= =========
## [v26.13.2] - 2025-04-14
### :bug: Bug Fixes
- [`9693dbd`](https://github.com/tobymao/sqlglot/commit/9693dbd18b98b2699cade738a254f71f2ee8ce74) - **clickhouse**: avoid superfluous parentheses in DISTINCT ON (...) *(commit by [@georgesittas](https://github.com/georgesittas))*
## [v26.13.1] - 2025-04-14
### :sparkles: New Features
- [`a762993`](https://github.com/tobymao/sqlglot/commit/a762993c53d7ae91a831a8be448010e17e60f497) - **generator**: unsupported warning for T-SQL query option *(PR [#4972](https://github.com/tobymao/sqlglot/pull/4972) by [@geooo109](https://github.com/geooo109))*
### :bug: Bug Fixes
- [`61bc01c`](https://github.com/tobymao/sqlglot/commit/61bc01ceec2f801490f3f1a571aee655c5109962) - **clickhouse**: allow string literal for clickhouse ON CLUSTER clause *(PR [#4971](https://github.com/tobymao/sqlglot/pull/4971) by [@lepfhty](https://github.com/lepfhty))*
- [`1353b79`](https://github.com/tobymao/sqlglot/commit/1353b79bd9810788a02163928b044fe038267078) - **Snowflake**: Enhance parity for FILE_FORMAT & CREDENTIALS in CREATE STAGE *(PR [#4969](https://github.com/tobymao/sqlglot/pull/4969) by [@whummer](https://github.com/whummer))*
## [v26.13.0] - 2025-04-11
### :boom: BREAKING CHANGES
- due to [`1df7f61`](https://github.com/tobymao/sqlglot/commit/1df7f611bc96616cb07950a80f6669d0bc331b0e) - refactor length_sql so it handles any type, not just varchar/blob *(PR [#4935](https://github.com/tobymao/sqlglot/pull/4935) by [@tekumara](https://github.com/tekumara))*:
refactor length_sql so it handles any type, not just varchar/blob (#4935)
- due to [`52719f3`](https://github.com/tobymao/sqlglot/commit/52719f37f6541e8ec9f66642ac23ed9015048092) - parse CREATE STAGE *(PR [#4947](https://github.com/tobymao/sqlglot/pull/4947) by [@tekumara](https://github.com/tekumara))*:
parse CREATE STAGE (#4947)
- due to [`fd39b30`](https://github.com/tobymao/sqlglot/commit/fd39b30209d068b787619b8137a105aca9c3e607) - parse CREATE FILE FORMAT *(PR [#4948](https://github.com/tobymao/sqlglot/pull/4948) by [@tekumara](https://github.com/tekumara))*:
parse CREATE FILE FORMAT (#4948)
- due to [`f835756`](https://github.com/tobymao/sqlglot/commit/f835756257f735643584b89e93693e8577744731) - Fix CREATE EXTERNAL TABLE properties *(PR [#4951](https://github.com/tobymao/sqlglot/pull/4951) by [@VaggelisD](https://github.com/VaggelisD))*:
Fix CREATE EXTERNAL TABLE properties (#4951)
- due to [`44b955b`](https://github.com/tobymao/sqlglot/commit/44b955bd537bfb8f5b6e84ecbcd5f6e3da852260) - Fix generation of exp.Values *(PR [#4930](https://github.com/tobymao/sqlglot/pull/4930) by [@VaggelisD](https://github.com/VaggelisD))*:
Fix generation of exp.Values (#4930)
- due to [`1f506b1`](https://github.com/tobymao/sqlglot/commit/1f506b186f1b954829195eefda318e231d474208) - support SHOW (ALL) TABLES *(PR [#4961](https://github.com/tobymao/sqlglot/pull/4961) by [@mscolnick](https://github.com/mscolnick))*:
support SHOW (ALL) TABLES (#4961)
- due to [`72cf4a4`](https://github.com/tobymao/sqlglot/commit/72cf4a4501a8d122041a28b71be5a41ffb53602a) - Add support for PIVOT multiple IN clauses *(PR [#4964](https://github.com/tobymao/sqlglot/pull/4964) by [@VaggelisD](https://github.com/VaggelisD))*:
Add support for PIVOT multiple IN clauses (#4964)
- due to [`400ea54`](https://github.com/tobymao/sqlglot/commit/400ea54d3a9cab256bfa5e496439bb9be6072d0b) - ensure JSON_FORMAT type is JSON when targeting Presto *(PR [#4968](https://github.com/tobymao/sqlglot/pull/4968) by [@georgesittas](https://github.com/georgesittas))*:
ensure JSON_FORMAT type is JSON when targeting Presto (#4968)
### :sparkles: New Features
- [`52719f3`](https://github.com/tobymao/sqlglot/commit/52719f37f6541e8ec9f66642ac23ed9015048092) - **snowflake**: parse CREATE STAGE *(PR [#4947](https://github.com/tobymao/sqlglot/pull/4947) by [@tekumara](https://github.com/tekumara))*
- [`fd39b30`](https://github.com/tobymao/sqlglot/commit/fd39b30209d068b787619b8137a105aca9c3e607) - **snowflake**: parse CREATE FILE FORMAT *(PR [#4948](https://github.com/tobymao/sqlglot/pull/4948) by [@tekumara](https://github.com/tekumara))*
- [`da9a6a1`](https://github.com/tobymao/sqlglot/commit/da9a6a1d56323319b87e9b193d12ad1c644b9239) - **snowflake**: parse SHOW STAGES *(PR [#4949](https://github.com/tobymao/sqlglot/pull/4949) by [@tekumara](https://github.com/tekumara))*
- [`bfdcdf0`](https://github.com/tobymao/sqlglot/commit/bfdcdf0afc0f4af3dacdfc3e8dca243793552b74) - **snowflake**: parse SHOW FILE FORMATS *(PR [#4950](https://github.com/tobymao/sqlglot/pull/4950) by [@tekumara](https://github.com/tekumara))*
- [`c591443`](https://github.com/tobymao/sqlglot/commit/c591443b6b2328780e08179144557e181db0cbb6) - **duckdb**: add support for GROUP clause in standard PIVOT syntax *(PR [#4953](https://github.com/tobymao/sqlglot/pull/4953) by [@georgesittas](https://github.com/georgesittas))*
- [`b011ee2`](https://github.com/tobymao/sqlglot/commit/b011ee2df0beaac75b982261a25d3e787dead54a) - **bigquery**: Add support for side & kind on set operators *(PR [#4959](https://github.com/tobymao/sqlglot/pull/4959) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *addresses issue [#4942](https://github.com/tobymao/sqlglot/issues/4942) opened by [@z3z1ma](https://github.com/z3z1ma)*
- [`1f506b1`](https://github.com/tobymao/sqlglot/commit/1f506b186f1b954829195eefda318e231d474208) - **duckdb**: support SHOW (ALL) TABLES *(PR [#4961](https://github.com/tobymao/sqlglot/pull/4961) by [@mscolnick](https://github.com/mscolnick))*
- :arrow_lower_right: *addresses issue [#4956](https://github.com/tobymao/sqlglot/issues/4956) opened by [@mscolnick](https://github.com/mscolnick)*
- [`ad5b595`](https://github.com/tobymao/sqlglot/commit/ad5b595049a16a27a7f249afea43dbcfcf43b5f4) - allow explicit aliasing in if(...) expressions *(PR [#4963](https://github.com/tobymao/sqlglot/pull/4963) by [@georgesittas](https://github.com/georgesittas))*
- [`72cf4a4`](https://github.com/tobymao/sqlglot/commit/72cf4a4501a8d122041a28b71be5a41ffb53602a) - **duckdb**: Add support for PIVOT multiple IN clauses *(PR [#4964](https://github.com/tobymao/sqlglot/pull/4964) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *addresses issue [#4944](https://github.com/tobymao/sqlglot/issues/4944) opened by [@nph](https://github.com/nph)*
- [`7bc5a21`](https://github.com/tobymao/sqlglot/commit/7bc5a217c3cc68d0cb1eaedc0c18f5188de80bf1) - **postgres**: support laterals with ordinality fixes [#4965](https://github.com/tobymao/sqlglot/pull/4965) *(PR [#4966](https://github.com/tobymao/sqlglot/pull/4966) by [@georgesittas](https://github.com/georgesittas))*
- [`400ea54`](https://github.com/tobymao/sqlglot/commit/400ea54d3a9cab256bfa5e496439bb9be6072d0b) - ensure JSON_FORMAT type is JSON when targeting Presto *(PR [#4968](https://github.com/tobymao/sqlglot/pull/4968) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#4967](https://github.com/tobymao/sqlglot/issues/4967) opened by [@jmsmdy](https://github.com/jmsmdy)*
### :bug: Bug Fixes
- [`7042603`](https://github.com/tobymao/sqlglot/commit/7042603ecb5693795b15219ec9cebf2f76032c03) - **optimizer**: Merge subqueries when inner query has name conflict with outer query *(PR [#4931](https://github.com/tobymao/sqlglot/pull/4931) by [@barakalon](https://github.com/barakalon))*
- [`1df7f61`](https://github.com/tobymao/sqlglot/commit/1df7f611bc96616cb07950a80f6669d0bc331b0e) - **duckdb**: refactor length_sql so it handles any type, not just varchar/blob *(PR [#4935](https://github.com/tobymao/sqlglot/pull/4935) by [@tekumara](https://github.com/tekumara))*
- :arrow_lower_right: *fixes issue [#4934](https://github.com/tobymao/sqlglot/issues/4934) opened by [@tekumara](https://github.com/tekumara)*
- [`09882e3`](https://github.com/tobymao/sqlglot/commit/09882e32f057670a9cbd97c1e5cf1a00c774b5d2) - **tsql**: remove assert call from _build_formatted_time *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`bf39a95`](https://github.com/tobymao/sqlglot/commit/bf39a95426ed6637e424da1be070cc9a8affc358) - **sqlite**: transpile double quoted PRIMARY KEY *(PR [#4941](https://github.com/tobymao/sqlglot/pull/4941) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *fixes issue [#4938](https://github.com/tobymao/sqlglot/issues/4938) opened by [@rgeronimi](https://github.com/rgeronimi)*
- [`f835756`](https://github.com/tobymao/sqlglot/commit/f835756257f735643584b89e93693e8577744731) - **snowflake**: Fix CREATE EXTERNAL TABLE properties *(PR [#4951](https://github.com/tobymao/sqlglot/pull/4951) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4945](https://github.com/tobymao/sqlglot/issues/4945) opened by [@tekumara](https://github.com/tekumara)*
- [`61ed971`](https://github.com/tobymao/sqlglot/commit/61ed971213c979c3777e57853bd6989bc169adb1) - **athena**: Correctly handle CTAS queries that contain Union's *(PR [#4955](https://github.com/tobymao/sqlglot/pull/4955) by [@erindru](https://github.com/erindru))*
- [`44b955b`](https://github.com/tobymao/sqlglot/commit/44b955bd537bfb8f5b6e84ecbcd5f6e3da852260) - **clickhouse**: Fix generation of exp.Values *(PR [#4930](https://github.com/tobymao/sqlglot/pull/4930) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4056](https://github.com/TobikoData/sqlmesh/issues/4056) opened by [@dnbnero](https://github.com/dnbnero)*
## [v26.12.0] - 2025-03-27 ## [v26.12.0] - 2025-03-27
### :boom: BREAKING CHANGES ### :boom: BREAKING CHANGES
- due to [`8a692b9`](https://github.com/tobymao/sqlglot/commit/8a692b9b5b7982ed54444bddfe974e5f629183ff) - support select...into #temp_table syntax *(PR [#4893](https://github.com/tobymao/sqlglot/pull/4893) by [@hhubbell](https://github.com/hhubbell))*: - due to [`8a692b9`](https://github.com/tobymao/sqlglot/commit/8a692b9b5b7982ed54444bddfe974e5f629183ff) - support select...into #temp_table syntax *(PR [#4893](https://github.com/tobymao/sqlglot/pull/4893) by [@hhubbell](https://github.com/hhubbell))*:
@ -6258,3 +6338,6 @@ Changelog
[v26.11.0]: https://github.com/tobymao/sqlglot/compare/v26.10.1...v26.11.0 [v26.11.0]: https://github.com/tobymao/sqlglot/compare/v26.10.1...v26.11.0
[v26.11.1]: https://github.com/tobymao/sqlglot/compare/v26.11.0...v26.11.1 [v26.11.1]: https://github.com/tobymao/sqlglot/compare/v26.11.0...v26.11.1
[v26.12.0]: https://github.com/tobymao/sqlglot/compare/v26.11.1...v26.12.0 [v26.12.0]: https://github.com/tobymao/sqlglot/compare/v26.11.1...v26.12.0
[v26.13.0]: https://github.com/tobymao/sqlglot/compare/v26.12.1...v26.13.0
[v26.13.1]: https://github.com/tobymao/sqlglot/compare/v26.13.0...v26.13.1
[v26.13.2]: https://github.com/tobymao/sqlglot/compare/v26.13.1...v26.13.2

2
docs/search.js
View file

File diff suppressed because one or more lines are too long

12
docs/sqlglot/_version.html
View file

@ -84,8 +84,8 @@
</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span> </span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span> </span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a> </span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a>
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;26.12.0&#39;</span> </span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;26.13.2&#39;</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">26</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> </span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">26</span><span class="p">,</span> <span class="mi">13</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
</span></pre></div> </span></pre></div>
@ -93,7 +93,7 @@
<section id="__version__"> <section id="__version__">
<div class="attr variable"> <div class="attr variable">
<span class="name">__version__</span><span class="annotation">: str</span> = <span class="name">__version__</span><span class="annotation">: str</span> =
<span class="default_value">&#39;26.12.0&#39;</span> <span class="default_value">&#39;26.13.2&#39;</span>
</div> </div>
@ -105,7 +105,7 @@
<section id="__version_tuple__"> <section id="__version_tuple__">
<div class="attr variable"> <div class="attr variable">
<span class="name">__version_tuple__</span><span class="annotation">: object</span> = <span class="name">__version_tuple__</span><span class="annotation">: object</span> =
<span class="default_value">(26, 12, 0)</span> <span class="default_value">(26, 13, 2)</span>
</div> </div>
@ -117,7 +117,7 @@
<section id="version"> <section id="version">
<div class="attr variable"> <div class="attr variable">
<span class="name">version</span><span class="annotation">: str</span> = <span class="name">version</span><span class="annotation">: str</span> =
<span class="default_value">&#39;26.12.0&#39;</span> <span class="default_value">&#39;26.13.2&#39;</span>
</div> </div>
@ -129,7 +129,7 @@
<section id="version_tuple"> <section id="version_tuple">
<div class="attr variable"> <div class="attr variable">
<span class="name">version_tuple</span><span class="annotation">: object</span> = <span class="name">version_tuple</span><span class="annotation">: object</span> =
<span class="default_value">(26, 12, 0)</span> <span class="default_value">(26, 13, 2)</span>
</div> </div>

58
docs/sqlglot/dialects.html
View file

@ -355,7 +355,7 @@ dialect implementations in order to understand how their various components can
<section id="Athena"> <section id="Athena">
<div class="attr variable"> <div class="attr variable">
<span class="name">Athena</span> = <span class="name">Athena</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454858791888&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264100748240&#39;&gt;</span>
</div> </div>
@ -367,7 +367,7 @@ dialect implementations in order to understand how their various components can
<section id="BigQuery"> <section id="BigQuery">
<div class="attr variable"> <div class="attr variable">
<span class="name">BigQuery</span> = <span class="name">BigQuery</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454858927520&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264100743344&#39;&gt;</span>
</div> </div>
@ -379,7 +379,7 @@ dialect implementations in order to understand how their various components can
<section id="ClickHouse"> <section id="ClickHouse">
<div class="attr variable"> <div class="attr variable">
<span class="name">ClickHouse</span> = <span class="name">ClickHouse</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454858929872&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264115191664&#39;&gt;</span>
</div> </div>
@ -391,7 +391,7 @@ dialect implementations in order to understand how their various components can
<section id="Databricks"> <section id="Databricks">
<div class="attr variable"> <div class="attr variable">
<span class="name">Databricks</span> = <span class="name">Databricks</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454845664752&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264112300352&#39;&gt;</span>
</div> </div>
@ -403,7 +403,7 @@ dialect implementations in order to understand how their various components can
<section id="Doris"> <section id="Doris">
<div class="attr variable"> <div class="attr variable">
<span class="name">Doris</span> = <span class="name">Doris</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454858558240&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264112292576&#39;&gt;</span>
</div> </div>
@ -415,7 +415,7 @@ dialect implementations in order to understand how their various components can
<section id="Drill"> <section id="Drill">
<div class="attr variable"> <div class="attr variable">
<span class="name">Drill</span> = <span class="name">Drill</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454850442672&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264108318464&#39;&gt;</span>
</div> </div>
@ -427,7 +427,7 @@ dialect implementations in order to understand how their various components can
<section id="Druid"> <section id="Druid">
<div class="attr variable"> <div class="attr variable">
<span class="name">Druid</span> = <span class="name">Druid</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454847995360&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264108317552&#39;&gt;</span>
</div> </div>
@ -439,7 +439,7 @@ dialect implementations in order to understand how their various components can
<section id="DuckDB"> <section id="DuckDB">
<div class="attr variable"> <div class="attr variable">
<span class="name">DuckDB</span> = <span class="name">DuckDB</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454847990128&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264107845824&#39;&gt;</span>
</div> </div>
@ -451,7 +451,7 @@ dialect implementations in order to understand how their various components can
<section id="Dune"> <section id="Dune">
<div class="attr variable"> <div class="attr variable">
<span class="name">Dune</span> = <span class="name">Dune</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454845577680&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264116948320&#39;&gt;</span>
</div> </div>
@ -463,7 +463,7 @@ dialect implementations in order to understand how their various components can
<section id="Hive"> <section id="Hive">
<div class="attr variable"> <div class="attr variable">
<span class="name">Hive</span> = <span class="name">Hive</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454858206336&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264116955808&#39;&gt;</span>
</div> </div>
@ -475,7 +475,7 @@ dialect implementations in order to understand how their various components can
<section id="Materialize"> <section id="Materialize">
<div class="attr variable"> <div class="attr variable">
<span class="name">Materialize</span> = <span class="name">Materialize</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454858281296&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264111251152&#39;&gt;</span>
</div> </div>
@ -487,7 +487,7 @@ dialect implementations in order to understand how their various components can
<section id="MySQL"> <section id="MySQL">
<div class="attr variable"> <div class="attr variable">
<span class="name">MySQL</span> = <span class="name">MySQL</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454858280144&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264115116528&#39;&gt;</span>
</div> </div>
@ -499,7 +499,7 @@ dialect implementations in order to understand how their various components can
<section id="Oracle"> <section id="Oracle">
<div class="attr variable"> <div class="attr variable">
<span class="name">Oracle</span> = <span class="name">Oracle</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454846419376&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264102972624&#39;&gt;</span>
</div> </div>
@ -511,7 +511,7 @@ dialect implementations in order to understand how their various components can
<section id="Postgres"> <section id="Postgres">
<div class="attr variable"> <div class="attr variable">
<span class="name">Postgres</span> = <span class="name">Postgres</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454846428928&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264111467168&#39;&gt;</span>
</div> </div>
@ -523,7 +523,7 @@ dialect implementations in order to understand how their various components can
<section id="Presto"> <section id="Presto">
<div class="attr variable"> <div class="attr variable">
<span class="name">Presto</span> = <span class="name">Presto</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454855261584&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264102706336&#39;&gt;</span>
</div> </div>
@ -535,7 +535,7 @@ dialect implementations in order to understand how their various components can
<section id="PRQL"> <section id="PRQL">
<div class="attr variable"> <div class="attr variable">
<span class="name">PRQL</span> = <span class="name">PRQL</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454850711264&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264102696928&#39;&gt;</span>
</div> </div>
@ -547,7 +547,7 @@ dialect implementations in order to understand how their various components can
<section id="Redshift"> <section id="Redshift">
<div class="attr variable"> <div class="attr variable">
<span class="name">Redshift</span> = <span class="name">Redshift</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454850716784&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264103185808&#39;&gt;</span>
</div> </div>
@ -559,7 +559,7 @@ dialect implementations in order to understand how their various components can
<section id="RisingWave"> <section id="RisingWave">
<div class="attr variable"> <div class="attr variable">
<span class="name">RisingWave</span> = <span class="name">RisingWave</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454858183376&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264103194400&#39;&gt;</span>
</div> </div>
@ -571,7 +571,7 @@ dialect implementations in order to understand how their various components can
<section id="Snowflake"> <section id="Snowflake">
<div class="attr variable"> <div class="attr variable">
<span class="name">Snowflake</span> = <span class="name">Snowflake</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454855514176&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264102785376&#39;&gt;</span>
</div> </div>
@ -583,7 +583,7 @@ dialect implementations in order to understand how their various components can
<section id="Spark"> <section id="Spark">
<div class="attr variable"> <div class="attr variable">
<span class="name">Spark</span> = <span class="name">Spark</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454855522336&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264102778560&#39;&gt;</span>
</div> </div>
@ -595,7 +595,7 @@ dialect implementations in order to understand how their various components can
<section id="Spark2"> <section id="Spark2">
<div class="attr variable"> <div class="attr variable">
<span class="name">Spark2</span> = <span class="name">Spark2</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454855108272&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264117325104&#39;&gt;</span>
</div> </div>
@ -607,7 +607,7 @@ dialect implementations in order to understand how their various components can
<section id="SQLite"> <section id="SQLite">
<div class="attr variable"> <div class="attr variable">
<span class="name">SQLite</span> = <span class="name">SQLite</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454859099536&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264117328656&#39;&gt;</span>
</div> </div>
@ -619,7 +619,7 @@ dialect implementations in order to understand how their various components can
<section id="StarRocks"> <section id="StarRocks">
<div class="attr variable"> <div class="attr variable">
<span class="name">StarRocks</span> = <span class="name">StarRocks</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454847968784&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264112789664&#39;&gt;</span>
</div> </div>
@ -631,7 +631,7 @@ dialect implementations in order to understand how their various components can
<section id="Tableau"> <section id="Tableau">
<div class="attr variable"> <div class="attr variable">
<span class="name">Tableau</span> = <span class="name">Tableau</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454847969312&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264112797488&#39;&gt;</span>
</div> </div>
@ -643,7 +643,7 @@ dialect implementations in order to understand how their various components can
<section id="Teradata"> <section id="Teradata">
<div class="attr variable"> <div class="attr variable">
<span class="name">Teradata</span> = <span class="name">Teradata</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454845998624&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264112400192&#39;&gt;</span>
</div> </div>
@ -655,7 +655,7 @@ dialect implementations in order to understand how their various components can
<section id="Trino"> <section id="Trino">
<div class="attr variable"> <div class="attr variable">
<span class="name">Trino</span> = <span class="name">Trino</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454846001408&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264112391696&#39;&gt;</span>
</div> </div>
@ -667,7 +667,7 @@ dialect implementations in order to understand how their various components can
<section id="TSQL"> <section id="TSQL">
<div class="attr variable"> <div class="attr variable">
<span class="name">TSQL</span> = <span class="name">TSQL</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454858272208&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264115133680&#39;&gt;</span>
</div> </div>
@ -679,7 +679,7 @@ dialect implementations in order to understand how their various components can
<section id="Dialect"> <section id="Dialect">
<div class="attr variable"> <div class="attr variable">
<span class="name">Dialect</span> = <span class="name">Dialect</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454847899456&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264105668192&#39;&gt;</span>
</div> </div>
@ -691,7 +691,7 @@ dialect implementations in order to understand how their various components can
<section id="Dialects"> <section id="Dialects">
<div class="attr variable"> <div class="attr variable">
<span class="name">Dialects</span> = <span class="name">Dialects</span> =
<span class="default_value">&lt;MagicMock id=&#39;140454858724864&#39;&gt;</span> <span class="default_value">&lt;MagicMock id=&#39;140264105670064&#39;&gt;</span>
</div> </div>

10
docs/sqlglot/dialects/athena.html
View file

File diff suppressed because one or more lines are too long

35
docs/sqlglot/dialects/bigquery.html
View file

File diff suppressed because one or more lines are too long

7059
docs/sqlglot/dialects/clickhouse.html
View file

File diff suppressed because one or more lines are too long

7
docs/sqlglot/dialects/databricks.html
View file

File diff suppressed because one or more lines are too long

7
docs/sqlglot/dialects/doris.html
View file

File diff suppressed because one or more lines are too long

7
docs/sqlglot/dialects/drill.html
View file

File diff suppressed because one or more lines are too long

5743
docs/sqlglot/dialects/duckdb.html
View file

File diff suppressed because one or more lines are too long

3
docs/sqlglot/dialects/dune.html
View file

File diff suppressed because one or more lines are too long

11
docs/sqlglot/dialects/hive.html
View file

File diff suppressed because one or more lines are too long

7
docs/sqlglot/dialects/materialize.html
View file

File diff suppressed because one or more lines are too long

21
docs/sqlglot/dialects/mysql.html
View file

File diff suppressed because one or more lines are too long

9
docs/sqlglot/dialects/oracle.html
View file

File diff suppressed because one or more lines are too long

3348
docs/sqlglot/dialects/postgres.html
View file

File diff suppressed because one or more lines are too long

3704
docs/sqlglot/dialects/presto.html
View file

File diff suppressed because one or more lines are too long

5
docs/sqlglot/dialects/prql.html
View file

File diff suppressed because one or more lines are too long

9
docs/sqlglot/dialects/redshift.html
View file

File diff suppressed because one or more lines are too long

7
docs/sqlglot/dialects/risingwave.html
View file

File diff suppressed because one or more lines are too long

5209
docs/sqlglot/dialects/snowflake.html
View file

File diff suppressed because one or more lines are too long

7
docs/sqlglot/dialects/spark.html
View file

File diff suppressed because one or more lines are too long

1343
docs/sqlglot/dialects/spark2.html
View file

File diff suppressed because one or more lines are too long

9
docs/sqlglot/dialects/sqlite.html
View file

File diff suppressed because one or more lines are too long

9
docs/sqlglot/dialects/starrocks.html
View file

File diff suppressed because one or more lines are too long

7
docs/sqlglot/dialects/tableau.html
View file

File diff suppressed because one or more lines are too long

11
docs/sqlglot/dialects/teradata.html
View file

File diff suppressed because one or more lines are too long

10
docs/sqlglot/dialects/trino.html
View file

File diff suppressed because one or more lines are too long

6528
docs/sqlglot/dialects/tsql.html
View file

File diff suppressed because one or more lines are too long

92
docs/sqlglot/executor.html
View file

@ -399,32 +399,34 @@
</span><span id="L-65"><a href="#L-65"><span class="linenos">65</span></a> </span><span id="L-65"><a href="#L-65"><span class="linenos">65</span></a>
</span><span id="L-66"><a href="#L-66"><span class="linenos">66</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span> </span><span id="L-66"><a href="#L-66"><span class="linenos">66</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos">67</span></a> <span class="n">value</span> <span class="o">=</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="n">column</span><span class="p">]</span> </span><span id="L-67"><a href="#L-67"><span class="linenos">67</span></a> <span class="n">value</span> <span class="o">=</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="n">column</span><span class="p">]</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos">68</span></a> <span class="n">column_type</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">convert</span><span class="p">(</span><span class="n">value</span><span class="p">))</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="nb">type</span><span class="p">(</span><span class="n">value</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> </span><span id="L-68"><a href="#L-68"><span class="linenos">68</span></a> <span class="n">column_type</span> <span class="o">=</span> <span class="p">(</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos">69</span></a> <span class="n">nested_set</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="p">[</span><span class="o">*</span><span class="n">keys</span><span class="p">,</span> <span class="n">column</span><span class="p">],</span> <span class="n">column_type</span><span class="p">)</span> </span><span id="L-69"><a href="#L-69"><span class="linenos">69</span></a> <span class="n">annotate_types</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">convert</span><span class="p">(</span><span class="n">value</span><span class="p">),</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="nb">type</span><span class="p">(</span><span class="n">value</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos">70</span></a> </span><span id="L-70"><a href="#L-70"><span class="linenos">70</span></a> <span class="p">)</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos">71</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span> </span><span id="L-71"><a href="#L-71"><span class="linenos">71</span></a> <span class="n">nested_set</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="p">[</span><span class="o">*</span><span class="n">keys</span><span class="p">,</span> <span class="n">column</span><span class="p">],</span> <span class="n">column_type</span><span class="p">)</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos">72</span></a> </span><span id="L-72"><a href="#L-72"><span class="linenos">72</span></a>
</span><span id="L-73"><a href="#L-73"><span class="linenos">73</span></a> <span class="k">if</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="ow">and</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="o">!=</span> <span class="n">schema</span><span class="o">.</span><span class="n">supported_table_args</span><span class="p">:</span> </span><span id="L-73"><a href="#L-73"><span class="linenos">73</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos">74</span></a> <span class="k">raise</span> <span class="n">ExecuteError</span><span class="p">(</span><span class="s2">&quot;Tables must support the same table args as schema&quot;</span><span class="p">)</span> </span><span id="L-74"><a href="#L-74"><span class="linenos">74</span></a>
</span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a> </span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a> <span class="k">if</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="ow">and</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="o">!=</span> <span class="n">schema</span><span class="o">.</span><span class="n">supported_table_args</span><span class="p">:</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos">76</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> </span><span id="L-76"><a href="#L-76"><span class="linenos">76</span></a> <span class="k">raise</span> <span class="n">ExecuteError</span><span class="p">(</span><span class="s2">&quot;Tables must support the same table args as schema&quot;</span><span class="p">)</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span> </span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a>
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a> <span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">infer_csv_schemas</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span> </span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a> <span class="p">)</span> </span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a> </span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a> <span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">infer_csv_schemas</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimization finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span> </span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a> <span class="p">)</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimized SQL: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span> </span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a>
</span><span id="L-83"><a href="#L-83"><span class="linenos">83</span></a> </span><span id="L-83"><a href="#L-83"><span class="linenos">83</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimization finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos">84</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="L-84"><a href="#L-84"><span class="linenos">84</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimized SQL: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
</span><span id="L-85"><a href="#L-85"><span class="linenos">85</span></a> </span><span id="L-85"><a href="#L-85"><span class="linenos">85</span></a>
</span><span id="L-86"><a href="#L-86"><span class="linenos">86</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Logical Plan: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span> </span><span id="L-86"><a href="#L-86"><span class="linenos">86</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-87"><a href="#L-87"><span class="linenos">87</span></a> </span><span id="L-87"><a href="#L-87"><span class="linenos">87</span></a>
</span><span id="L-88"><a href="#L-88"><span class="linenos">88</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> </span><span id="L-88"><a href="#L-88"><span class="linenos">88</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Logical Plan: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos">89</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span> </span><span id="L-89"><a href="#L-89"><span class="linenos">89</span></a>
</span><span id="L-90"><a href="#L-90"><span class="linenos">90</span></a> </span><span id="L-90"><a href="#L-90"><span class="linenos">90</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos">91</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Query finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span> </span><span id="L-91"><a href="#L-91"><span class="linenos">91</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos">92</span></a> </span><span id="L-92"><a href="#L-92"><span class="linenos">92</span></a>
</span><span id="L-93"><a href="#L-93"><span class="linenos">93</span></a> <span class="k">return</span> <span class="n">result</span> </span><span id="L-93"><a href="#L-93"><span class="linenos">93</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Query finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos">94</span></a>
</span><span id="L-95"><a href="#L-95"><span class="linenos">95</span></a> <span class="k">return</span> <span class="n">result</span>
</span></pre></div> </span></pre></div>
@ -489,32 +491,34 @@
</span><span id="execute-66"><a href="#execute-66"><span class="linenos">66</span></a> </span><span id="execute-66"><a href="#execute-66"><span class="linenos">66</span></a>
</span><span id="execute-67"><a href="#execute-67"><span class="linenos">67</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span> </span><span id="execute-67"><a href="#execute-67"><span class="linenos">67</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">table</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="execute-68"><a href="#execute-68"><span class="linenos">68</span></a> <span class="n">value</span> <span class="o">=</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="n">column</span><span class="p">]</span> </span><span id="execute-68"><a href="#execute-68"><span class="linenos">68</span></a> <span class="n">value</span> <span class="o">=</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="n">column</span><span class="p">]</span>
</span><span id="execute-69"><a href="#execute-69"><span class="linenos">69</span></a> <span class="n">column_type</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">convert</span><span class="p">(</span><span class="n">value</span><span class="p">))</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="nb">type</span><span class="p">(</span><span class="n">value</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> </span><span id="execute-69"><a href="#execute-69"><span class="linenos">69</span></a> <span class="n">column_type</span> <span class="o">=</span> <span class="p">(</span>
</span><span id="execute-70"><a href="#execute-70"><span class="linenos">70</span></a> <span class="n">nested_set</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="p">[</span><span class="o">*</span><span class="n">keys</span><span class="p">,</span> <span class="n">column</span><span class="p">],</span> <span class="n">column_type</span><span class="p">)</span> </span><span id="execute-70"><a href="#execute-70"><span class="linenos">70</span></a> <span class="n">annotate_types</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">convert</span><span class="p">(</span><span class="n">value</span><span class="p">),</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="nb">type</span><span class="p">(</span><span class="n">value</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span>
</span><span id="execute-71"><a href="#execute-71"><span class="linenos">71</span></a> </span><span id="execute-71"><a href="#execute-71"><span class="linenos">71</span></a> <span class="p">)</span>
</span><span id="execute-72"><a href="#execute-72"><span class="linenos">72</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span> </span><span id="execute-72"><a href="#execute-72"><span class="linenos">72</span></a> <span class="n">nested_set</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="p">[</span><span class="o">*</span><span class="n">keys</span><span class="p">,</span> <span class="n">column</span><span class="p">],</span> <span class="n">column_type</span><span class="p">)</span>
</span><span id="execute-73"><a href="#execute-73"><span class="linenos">73</span></a> </span><span id="execute-73"><a href="#execute-73"><span class="linenos">73</span></a>
</span><span id="execute-74"><a href="#execute-74"><span class="linenos">74</span></a> <span class="k">if</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="ow">and</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="o">!=</span> <span class="n">schema</span><span class="o">.</span><span class="n">supported_table_args</span><span class="p">:</span> </span><span id="execute-74"><a href="#execute-74"><span class="linenos">74</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
</span><span id="execute-75"><a href="#execute-75"><span class="linenos">75</span></a> <span class="k">raise</span> <span class="n">ExecuteError</span><span class="p">(</span><span class="s2">&quot;Tables must support the same table args as schema&quot;</span><span class="p">)</span> </span><span id="execute-75"><a href="#execute-75"><span class="linenos">75</span></a>
</span><span id="execute-76"><a href="#execute-76"><span class="linenos">76</span></a> </span><span id="execute-76"><a href="#execute-76"><span class="linenos">76</span></a> <span class="k">if</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="ow">and</span> <span class="n">tables_</span><span class="o">.</span><span class="n">supported_table_args</span> <span class="o">!=</span> <span class="n">schema</span><span class="o">.</span><span class="n">supported_table_args</span><span class="p">:</span>
</span><span id="execute-77"><a href="#execute-77"><span class="linenos">77</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> </span><span id="execute-77"><a href="#execute-77"><span class="linenos">77</span></a> <span class="k">raise</span> <span class="n">ExecuteError</span><span class="p">(</span><span class="s2">&quot;Tables must support the same table args as schema&quot;</span><span class="p">)</span>
</span><span id="execute-78"><a href="#execute-78"><span class="linenos">78</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span> </span><span id="execute-78"><a href="#execute-78"><span class="linenos">78</span></a>
</span><span id="execute-79"><a href="#execute-79"><span class="linenos">79</span></a> <span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">infer_csv_schemas</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span> </span><span id="execute-79"><a href="#execute-79"><span class="linenos">79</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
</span><span id="execute-80"><a href="#execute-80"><span class="linenos">80</span></a> <span class="p">)</span> </span><span id="execute-80"><a href="#execute-80"><span class="linenos">80</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span>
</span><span id="execute-81"><a href="#execute-81"><span class="linenos">81</span></a> </span><span id="execute-81"><a href="#execute-81"><span class="linenos">81</span></a> <span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">infer_csv_schemas</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span>
</span><span id="execute-82"><a href="#execute-82"><span class="linenos">82</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimization finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span> </span><span id="execute-82"><a href="#execute-82"><span class="linenos">82</span></a> <span class="p">)</span>
</span><span id="execute-83"><a href="#execute-83"><span class="linenos">83</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimized SQL: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span> </span><span id="execute-83"><a href="#execute-83"><span class="linenos">83</span></a>
</span><span id="execute-84"><a href="#execute-84"><span class="linenos">84</span></a> </span><span id="execute-84"><a href="#execute-84"><span class="linenos">84</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimization finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
</span><span id="execute-85"><a href="#execute-85"><span class="linenos">85</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="execute-85"><a href="#execute-85"><span class="linenos">85</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimized SQL: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
</span><span id="execute-86"><a href="#execute-86"><span class="linenos">86</span></a> </span><span id="execute-86"><a href="#execute-86"><span class="linenos">86</span></a>
</span><span id="execute-87"><a href="#execute-87"><span class="linenos">87</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Logical Plan: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span> </span><span id="execute-87"><a href="#execute-87"><span class="linenos">87</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="execute-88"><a href="#execute-88"><span class="linenos">88</span></a> </span><span id="execute-88"><a href="#execute-88"><span class="linenos">88</span></a>
</span><span id="execute-89"><a href="#execute-89"><span class="linenos">89</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> </span><span id="execute-89"><a href="#execute-89"><span class="linenos">89</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Logical Plan: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span>
</span><span id="execute-90"><a href="#execute-90"><span class="linenos">90</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span> </span><span id="execute-90"><a href="#execute-90"><span class="linenos">90</span></a>
</span><span id="execute-91"><a href="#execute-91"><span class="linenos">91</span></a> </span><span id="execute-91"><a href="#execute-91"><span class="linenos">91</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
</span><span id="execute-92"><a href="#execute-92"><span class="linenos">92</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Query finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span> </span><span id="execute-92"><a href="#execute-92"><span class="linenos">92</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span>
</span><span id="execute-93"><a href="#execute-93"><span class="linenos">93</span></a> </span><span id="execute-93"><a href="#execute-93"><span class="linenos">93</span></a>
</span><span id="execute-94"><a href="#execute-94"><span class="linenos">94</span></a> <span class="k">return</span> <span class="n">result</span> </span><span id="execute-94"><a href="#execute-94"><span class="linenos">94</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Query finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
</span><span id="execute-95"><a href="#execute-95"><span class="linenos">95</span></a>
</span><span id="execute-96"><a href="#execute-96"><span class="linenos">96</span></a> <span class="k">return</span> <span class="n">result</span>
</span></pre></div> </span></pre></div>

22752
docs/sqlglot/expressions.html
View file

File diff suppressed because it is too large Load diff

25737
docs/sqlglot/generator.html
View file

File diff suppressed because one or more lines are too long

1838
docs/sqlglot/helper.html
View file

File diff suppressed because it is too large Load diff

2
docs/sqlglot/jsonpath.html
View file

@ -641,7 +641,7 @@
<div class="attr variable"> <div class="attr variable">
<span class="name">ALL_JSON_PATH_PARTS</span> = <span class="name">ALL_JSON_PATH_PARTS</span> =
<input id="ALL_JSON_PATH_PARTS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="ALL_JSON_PATH_PARTS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>&#39;&gt;}</span> <label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>&#39;&gt;}</span>
</div> </div>

4
docs/sqlglot/optimizer/annotate_types.html
View file

File diff suppressed because one or more lines are too long

14
docs/sqlglot/optimizer/canonicalize.html
View file

@ -102,7 +102,7 @@
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a> </span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a>
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">_canonicalize</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> </span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">_canonicalize</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">add_text_to_concat</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">add_text_to_concat</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">replace_date_funcs</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">replace_date_funcs</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">coerce_type</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">.</span><span class="n">PROMOTE_TO_INFERRED_DATETIME_TYPE</span><span class="p">)</span> </span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">coerce_type</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">.</span><span class="n">PROMOTE_TO_INFERRED_DATETIME_TYPE</span><span class="p">)</span>
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">ensure_bools</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">_replace_int_predicate</span><span class="p">)</span> </span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">ensure_bools</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">_replace_int_predicate</span><span class="p">)</span>
@ -118,7 +118,7 @@
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="k">return</span> <span class="n">node</span> </span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> </span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> </span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a><span class="k">def</span><span class="w"> </span><span class="nf">replace_date_funcs</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> </span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a><span class="k">def</span><span class="w"> </span><span class="nf">replace_date_funcs</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> <span class="k">if</span> <span class="p">(</span> </span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TsOrDsToDate</span><span class="p">))</span> </span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TsOrDsToDate</span><span class="p">))</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> </span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span>
@ -131,7 +131,7 @@
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span> </span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.optimizer.annotate_types</span><span class="w"> </span><span class="kn">import</span> <span class="n">annotate_types</span> </span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.optimizer.annotate_types</span><span class="w"> </span><span class="kn">import</span> <span class="n">annotate_types</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> </span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">)</span> </span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span> </span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> </span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="k">return</span> <span class="n">node</span> </span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="k">return</span> <span class="n">node</span>
@ -325,7 +325,7 @@
</span><span id="canonicalize-24"><a href="#canonicalize-24"><span class="linenos">24</span></a> </span><span id="canonicalize-24"><a href="#canonicalize-24"><span class="linenos">24</span></a>
</span><span id="canonicalize-25"><a href="#canonicalize-25"><span class="linenos">25</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">_canonicalize</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> </span><span id="canonicalize-25"><a href="#canonicalize-25"><span class="linenos">25</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">_canonicalize</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="canonicalize-26"><a href="#canonicalize-26"><span class="linenos">26</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">add_text_to_concat</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="canonicalize-26"><a href="#canonicalize-26"><span class="linenos">26</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">add_text_to_concat</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="canonicalize-27"><a href="#canonicalize-27"><span class="linenos">27</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">replace_date_funcs</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="canonicalize-27"><a href="#canonicalize-27"><span class="linenos">27</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">replace_date_funcs</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="canonicalize-28"><a href="#canonicalize-28"><span class="linenos">28</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">coerce_type</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">.</span><span class="n">PROMOTE_TO_INFERRED_DATETIME_TYPE</span><span class="p">)</span> </span><span id="canonicalize-28"><a href="#canonicalize-28"><span class="linenos">28</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">coerce_type</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">.</span><span class="n">PROMOTE_TO_INFERRED_DATETIME_TYPE</span><span class="p">)</span>
</span><span id="canonicalize-29"><a href="#canonicalize-29"><span class="linenos">29</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="canonicalize-29"><a href="#canonicalize-29"><span class="linenos">29</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="canonicalize-30"><a href="#canonicalize-30"><span class="linenos">30</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">ensure_bools</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">_replace_int_predicate</span><span class="p">)</span> </span><span id="canonicalize-30"><a href="#canonicalize-30"><span class="linenos">30</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">ensure_bools</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">_replace_int_predicate</span><span class="p">)</span>
@ -376,13 +376,13 @@ conversions rely on type inference.</p>
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">replace_date_funcs</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">node</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span></span><span class="return-annotation">) -> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span>:</span></span> <span class="name">replace_date_funcs</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">node</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n"><a href="../dialects.html#Dialect">sqlglot.dialects.Dialect</a></span><span class="p">,</span> <span class="n">Type</span><span class="p">[</span><span class="n"><a href="../dialects.html#Dialect">sqlglot.dialects.Dialect</a></span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span>:</span></span>
<label class="view-source-button" for="replace_date_funcs-view-source"><span>View Source</span></label> <label class="view-source-button" for="replace_date_funcs-view-source"><span>View Source</span></label>
</div> </div>
<a class="headerlink" href="#replace_date_funcs"></a> <a class="headerlink" href="#replace_date_funcs"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="replace_date_funcs-43"><a href="#replace_date_funcs-43"><span class="linenos">43</span></a><span class="k">def</span><span class="w"> </span><span class="nf">replace_date_funcs</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span> <div class="pdoc-code codehilite"><pre><span></span><span id="replace_date_funcs-43"><a href="#replace_date_funcs-43"><span class="linenos">43</span></a><span class="k">def</span><span class="w"> </span><span class="nf">replace_date_funcs</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="replace_date_funcs-44"><a href="#replace_date_funcs-44"><span class="linenos">44</span></a> <span class="k">if</span> <span class="p">(</span> </span><span id="replace_date_funcs-44"><a href="#replace_date_funcs-44"><span class="linenos">44</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="replace_date_funcs-45"><a href="#replace_date_funcs-45"><span class="linenos">45</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TsOrDsToDate</span><span class="p">))</span> </span><span id="replace_date_funcs-45"><a href="#replace_date_funcs-45"><span class="linenos">45</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TsOrDsToDate</span><span class="p">))</span>
</span><span id="replace_date_funcs-46"><a href="#replace_date_funcs-46"><span class="linenos">46</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span> </span><span id="replace_date_funcs-46"><a href="#replace_date_funcs-46"><span class="linenos">46</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span>
@ -395,7 +395,7 @@ conversions rely on type inference.</p>
</span><span id="replace_date_funcs-53"><a href="#replace_date_funcs-53"><span class="linenos">53</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span> </span><span id="replace_date_funcs-53"><a href="#replace_date_funcs-53"><span class="linenos">53</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
</span><span id="replace_date_funcs-54"><a href="#replace_date_funcs-54"><span class="linenos">54</span></a> <span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.optimizer.annotate_types</span><span class="w"> </span><span class="kn">import</span> <span class="n">annotate_types</span> </span><span id="replace_date_funcs-54"><a href="#replace_date_funcs-54"><span class="linenos">54</span></a> <span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.optimizer.annotate_types</span><span class="w"> </span><span class="kn">import</span> <span class="n">annotate_types</span>
</span><span id="replace_date_funcs-55"><a href="#replace_date_funcs-55"><span class="linenos">55</span></a> </span><span id="replace_date_funcs-55"><a href="#replace_date_funcs-55"><span class="linenos">55</span></a>
</span><span id="replace_date_funcs-56"><a href="#replace_date_funcs-56"><span class="linenos">56</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">)</span> </span><span id="replace_date_funcs-56"><a href="#replace_date_funcs-56"><span class="linenos">56</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="replace_date_funcs-57"><a href="#replace_date_funcs-57"><span class="linenos">57</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span> </span><span id="replace_date_funcs-57"><a href="#replace_date_funcs-57"><span class="linenos">57</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
</span><span id="replace_date_funcs-58"><a href="#replace_date_funcs-58"><span class="linenos">58</span></a> </span><span id="replace_date_funcs-58"><a href="#replace_date_funcs-58"><span class="linenos">58</span></a>
</span><span id="replace_date_funcs-59"><a href="#replace_date_funcs-59"><span class="linenos">59</span></a> <span class="k">return</span> <span class="n">node</span> </span><span id="replace_date_funcs-59"><a href="#replace_date_funcs-59"><span class="linenos">59</span></a> <span class="k">return</span> <span class="n">node</span>

14
docs/sqlglot/optimizer/merge_subqueries.html
View file

@ -307,12 +307,12 @@
</span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a> <span class="n">source</span><span class="p">,</span> <span class="n">_</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">conflict</span><span class="p">]</span> </span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a> <span class="n">source</span><span class="p">,</span> <span class="n">_</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">conflict</span><span class="p">]</span>
</span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a> <span class="n">new_alias</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_name</span><span class="p">)</span> </span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a> <span class="n">new_alias</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_name</span><span class="p">)</span>
</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a> </span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a>
</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span> </span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">new_alias</span><span class="p">))</span> </span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">new_alias</span><span class="p">)</span>
</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span> </span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">new_alias</span><span class="p">)</span> </span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a> <span class="n">source</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">new_alias</span><span class="p">))</span>
</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span> </span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a> <span class="n">source</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">new_alias</span><span class="p">))</span> </span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a> <span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">new_alias</span><span class="p">))</span>
</span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a> </span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a>
</span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">source_columns</span><span class="p">(</span><span class="n">conflict</span><span class="p">):</span> </span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">source_columns</span><span class="p">(</span><span class="n">conflict</span><span class="p">):</span>
</span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;table&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_name</span><span class="p">))</span> </span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;table&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_name</span><span class="p">))</span>
@ -580,7 +580,7 @@ queries if it would result in multiple table selects in a single query:</p>
<div class="attr variable"> <div class="attr variable">
<span class="name">UNMERGABLE_ARGS</span> = <span class="name">UNMERGABLE_ARGS</span> =
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;prewhere&#39;, &#39;locks&#39;, &#39;with&#39;, &#39;sample&#39;, &#39;sort&#39;, &#39;settings&#39;, &#39;format&#39;, &#39;having&#39;, &#39;cluster&#39;, &#39;match&#39;, &#39;windows&#39;, &#39;group&#39;, &#39;qualify&#39;, &#39;laterals&#39;, &#39;kind&#39;, &#39;limit&#39;, &#39;pivots&#39;, &#39;offset&#39;, &#39;connect&#39;, &#39;into&#39;, &#39;operation_modifiers&#39;, &#39;distinct&#39;, &#39;distribute&#39;, &#39;options&#39;}</span> <label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;into&#39;, &#39;limit&#39;, &#39;sample&#39;, &#39;with&#39;, &#39;distinct&#39;, &#39;laterals&#39;, &#39;group&#39;, &#39;distribute&#39;, &#39;cluster&#39;, &#39;kind&#39;, &#39;format&#39;, &#39;prewhere&#39;, &#39;having&#39;, &#39;qualify&#39;, &#39;match&#39;, &#39;connect&#39;, &#39;operation_modifiers&#39;, &#39;offset&#39;, &#39;options&#39;, &#39;locks&#39;, &#39;pivots&#39;, &#39;settings&#39;, &#39;sort&#39;, &#39;windows&#39;}</span>
</div> </div>

348
docs/sqlglot/optimizer/pushdown_projections.html
View file

@ -113,101 +113,107 @@
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">parent_selections</span> <span class="o">=</span> <span class="p">{</span><span class="n">SELECT_ALL</span><span class="p">}</span> </span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">parent_selections</span> <span class="o">=</span> <span class="p">{</span><span class="n">SELECT_ALL</span><span class="p">}</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> </span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">SetOperation</span><span class="p">):</span> </span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">SetOperation</span><span class="p">):</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="n">left</span><span class="p">,</span> <span class="n">right</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">union_scopes</span> </span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="n">set_op</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span> <span class="o">!=</span> <span class="nb">len</span><span class="p">(</span><span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span> </span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="p">(</span><span class="n">set_op</span><span class="o">.</span><span class="n">kind</span> <span class="ow">or</span> <span class="n">set_op</span><span class="o">.</span><span class="n">side</span><span class="p">):</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">scope_sql</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">()</span> </span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="c1"># Do not optimize this set operation if it&#39;s using the BigQuery specific</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;Invalid set operation due to column mismatch: </span><span class="si">{</span><span class="n">scope_sql</span><span class="si">}</span><span class="s2">.&quot;</span><span class="p">)</span> </span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="c1"># kind / side syntax (e.g INNER UNION ALL BY NAME) which changes the semantics of the operation</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> </span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">left</span><span class="p">,</span> <span class="n">right</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">union_scopes</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">left</span><span class="p">]</span> <span class="o">=</span> <span class="n">parent_selections</span> </span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span> <span class="o">!=</span> <span class="nb">len</span><span class="p">(</span><span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> </span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">scope_sql</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">()</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="k">if</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span> </span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="n">parent_selections</span> </span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="sa">f</span><span class="s2">&quot;Invalid set operation due to column mismatch: </span><span class="si">{</span><span class="n">scope_sql</span><span class="si">}</span><span class="s2">.&quot;</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="k">elif</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span> </span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="p">)</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;by_name&quot;</span><span class="p">):</span> </span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">left</span><span class="p">]</span> </span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">left</span><span class="p">]</span> <span class="o">=</span> <span class="n">parent_selections</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span> </span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="k">if</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">alias_or_name</span> </span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="n">parent_selections</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">select</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span> </span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="k">elif</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="k">if</span> <span class="n">SELECT_ALL</span> <span class="ow">in</span> <span class="n">parent_selections</span> </span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;by_name&quot;</span><span class="p">):</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="ow">or</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">in</span> <span class="n">parent_selections</span> </span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">left</span><span class="p">]</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="p">]</span> </span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> </span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span> </span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">alias_or_name</span>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="k">if</span> <span class="n">remove_unused_selections</span><span class="p">:</span> </span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">select</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="n">_remove_unused_selections</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">parent_selections</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">alias_count</span><span class="p">)</span> </span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="k">if</span> <span class="n">SELECT_ALL</span> <span class="ow">in</span> <span class="n">parent_selections</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> </span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="ow">or</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">in</span> <span class="n">parent_selections</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">is_star</span><span class="p">:</span> </span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="p">]</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="k">continue</span> </span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> </span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="c1"># Group columns by source name</span> </span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="k">if</span> <span class="n">remove_unused_selections</span><span class="p">:</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="n">selects</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">set</span><span class="p">)</span> </span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="n">_remove_unused_selections</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">parent_selections</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">alias_count</span><span class="p">)</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span> </span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">table</span> </span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">is_star</span><span class="p">:</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="n">col_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">name</span> </span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">continue</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="n">selects</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span> </span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> </span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="c1"># Group columns by source name</span>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="c1"># Push the selected columns down to the next scope</span> </span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="n">selects</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">set</span><span class="p">)</span>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">source</span><span class="p">)</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span> </span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span> </span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">table</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">SELECT_ALL</span><span class="p">}</span> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">pivots</span> <span class="k">else</span> <span class="n">selects</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="ow">or</span> <span class="nb">set</span><span class="p">()</span> </span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">col_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">name</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">source</span><span class="p">]</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span> </span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="n">selects</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> </span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">alias_column_names</span> </span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="c1"># Push the selected columns down to the next scope</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">if</span> <span class="n">column_aliases</span><span class="p">:</span> </span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">source</span><span class="p">)</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">source_column_alias_count</span><span class="p">[</span><span class="n">source</span><span class="p">]</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">column_aliases</span><span class="p">)</span> </span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> </span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">SELECT_ALL</span><span class="p">}</span> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">pivots</span> <span class="k">else</span> <span class="n">selects</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="ow">or</span> <span class="nb">set</span><span class="p">()</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">source</span><span class="p">]</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> </span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> </span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">alias_column_names</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_remove_unused_selections</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">parent_selections</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">alias_count</span><span class="p">):</span> </span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="k">if</span> <span class="n">column_aliases</span><span class="p">:</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">order</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;order&quot;</span><span class="p">)</span> </span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">source_column_alias_count</span><span class="p">[</span><span class="n">source</span><span class="p">]</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">column_aliases</span><span class="p">)</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> </span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">if</span> <span class="n">order</span><span class="p">:</span> </span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="c1"># Assume columns without a qualified table are references to output columns</span> </span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="n">order_refs</span> <span class="o">=</span> <span class="p">{</span><span class="n">c</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">order</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">c</span><span class="o">.</span><span class="n">table</span><span class="p">}</span> </span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_remove_unused_selections</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">parent_selections</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">alias_count</span><span class="p">):</span>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="n">order_refs</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span> </span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="n">order</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;order&quot;</span><span class="p">)</span>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> </span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">new_selections</span> <span class="o">=</span> <span class="p">[]</span> </span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="k">if</span> <span class="n">order</span><span class="p">:</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">removed</span> <span class="o">=</span> <span class="kc">False</span> </span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="c1"># Assume columns without a qualified table are references to output columns</span>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">star</span> <span class="o">=</span> <span class="kc">False</span> </span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">order_refs</span> <span class="o">=</span> <span class="p">{</span><span class="n">c</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">order</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">c</span><span class="o">.</span><span class="n">table</span><span class="p">}</span>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="n">is_agg</span> <span class="o">=</span> <span class="kc">False</span> </span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> </span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="n">order_refs</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="n">select_all</span> <span class="o">=</span> <span class="n">SELECT_ALL</span> <span class="ow">in</span> <span class="n">parent_selections</span> </span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> </span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="n">new_selections</span> <span class="o">=</span> <span class="p">[]</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="k">for</span> <span class="n">selection</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span> </span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">removed</span> <span class="o">=</span> <span class="kc">False</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">selection</span><span class="o">.</span><span class="n">alias_or_name</span> </span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">star</span> <span class="o">=</span> <span class="kc">False</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> </span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">is_agg</span> <span class="o">=</span> <span class="kc">False</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="k">if</span> <span class="n">select_all</span> <span class="ow">or</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">parent_selections</span> <span class="ow">or</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">order_refs</span> <span class="ow">or</span> <span class="n">alias_count</span> <span class="o">&gt;</span> <span class="mi">0</span><span class="p">:</span> </span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">new_selections</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">selection</span><span class="p">)</span> </span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">select_all</span> <span class="o">=</span> <span class="n">SELECT_ALL</span> <span class="ow">in</span> <span class="n">parent_selections</span>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="n">alias_count</span> <span class="o">-=</span> <span class="mi">1</span> </span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">for</span> <span class="n">selection</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="k">if</span> <span class="n">selection</span><span class="o">.</span><span class="n">is_star</span><span class="p">:</span> </span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">selection</span><span class="o">.</span><span class="n">alias_or_name</span>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">star</span> <span class="o">=</span> <span class="kc">True</span> </span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">removed</span> <span class="o">=</span> <span class="kc">True</span> </span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">if</span> <span class="n">select_all</span> <span class="ow">or</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">parent_selections</span> <span class="ow">or</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">order_refs</span> <span class="ow">or</span> <span class="n">alias_count</span> <span class="o">&gt;</span> <span class="mi">0</span><span class="p">:</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> </span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">new_selections</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">selection</span><span class="p">)</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">is_agg</span> <span class="ow">and</span> <span class="n">selection</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">):</span> </span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="n">alias_count</span> <span class="o">-=</span> <span class="mi">1</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">is_agg</span> <span class="o">=</span> <span class="kc">True</span> </span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> </span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">if</span> <span class="n">selection</span><span class="o">.</span><span class="n">is_star</span><span class="p">:</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">if</span> <span class="n">star</span><span class="p">:</span> </span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">star</span> <span class="o">=</span> <span class="kc">True</span>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="n">resolver</span> <span class="o">=</span> <span class="n">Resolver</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">schema</span><span class="p">)</span> </span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="n">removed</span> <span class="o">=</span> <span class="kc">True</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">names</span> <span class="o">=</span> <span class="p">{</span><span class="n">s</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">new_selections</span><span class="p">}</span> </span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> </span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">is_agg</span> <span class="ow">and</span> <span class="n">selection</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">):</span>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">parent_selections</span><span class="p">):</span> </span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="n">is_agg</span> <span class="o">=</span> <span class="kc">True</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">names</span><span class="p">:</span> </span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="n">new_selections</span><span class="o">.</span><span class="n">append</span><span class="p">(</span> </span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="k">if</span> <span class="n">star</span><span class="p">:</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="n">resolver</span><span class="o">.</span><span class="n">get_table</span><span class="p">(</span><span class="n">name</span><span class="p">)),</span> <span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> </span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">resolver</span> <span class="o">=</span> <span class="n">Resolver</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">schema</span><span class="p">)</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="p">)</span> </span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="n">names</span> <span class="o">=</span> <span class="p">{</span><span class="n">s</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">new_selections</span><span class="p">}</span>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> </span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="c1"># If there are no remaining selections, just select a single constant</span> </span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">parent_selections</span><span class="p">):</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">new_selections</span><span class="p">:</span> </span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">names</span><span class="p">:</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">new_selections</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">default_selection</span><span class="p">(</span><span class="n">is_agg</span><span class="p">))</span> </span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">new_selections</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> </span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="n">resolver</span><span class="o">.</span><span class="n">get_table</span><span class="p">(</span><span class="n">name</span><span class="p">)),</span> <span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="o">*</span><span class="n">new_selections</span><span class="p">,</span> <span class="n">append</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> </span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="p">)</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> </span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="k">if</span> <span class="n">removed</span><span class="p">:</span> </span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="c1"># If there are no remaining selections, just select a single constant</span>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">clear_cache</span><span class="p">()</span> </span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">new_selections</span><span class="p">:</span>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">new_selections</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">default_selection</span><span class="p">(</span><span class="n">is_agg</span><span class="p">))</span>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="o">*</span><span class="n">new_selections</span><span class="p">,</span> <span class="n">append</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a>
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="k">if</span> <span class="n">removed</span><span class="p">:</span>
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">clear_cache</span><span class="p">()</span>
</span></pre></div> </span></pre></div>
@ -254,85 +260,91 @@
</div> </div>
<a class="headerlink" href="#pushdown_projections"></a> <a class="headerlink" href="#pushdown_projections"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown_projections-19"><a href="#pushdown_projections-19"><span class="linenos">19</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown_projections</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">remove_unused_selections</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span> <div class="pdoc-code codehilite"><pre><span></span><span id="pushdown_projections-19"><a href="#pushdown_projections-19"><span class="linenos"> 19</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown_projections</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">remove_unused_selections</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span>
</span><span id="pushdown_projections-20"><a href="#pushdown_projections-20"><span class="linenos">20</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="pushdown_projections-20"><a href="#pushdown_projections-20"><span class="linenos"> 20</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="pushdown_projections-21"><a href="#pushdown_projections-21"><span class="linenos">21</span></a><span class="sd"> Rewrite sqlglot AST to remove unused columns projections.</span> </span><span id="pushdown_projections-21"><a href="#pushdown_projections-21"><span class="linenos"> 21</span></a><span class="sd"> Rewrite sqlglot AST to remove unused columns projections.</span>
</span><span id="pushdown_projections-22"><a href="#pushdown_projections-22"><span class="linenos">22</span></a> </span><span id="pushdown_projections-22"><a href="#pushdown_projections-22"><span class="linenos"> 22</span></a>
</span><span id="pushdown_projections-23"><a href="#pushdown_projections-23"><span class="linenos">23</span></a><span class="sd"> Example:</span> </span><span id="pushdown_projections-23"><a href="#pushdown_projections-23"><span class="linenos"> 23</span></a><span class="sd"> Example:</span>
</span><span id="pushdown_projections-24"><a href="#pushdown_projections-24"><span class="linenos">24</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span> </span><span id="pushdown_projections-24"><a href="#pushdown_projections-24"><span class="linenos"> 24</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="pushdown_projections-25"><a href="#pushdown_projections-25"><span class="linenos">25</span></a><span class="sd"> &gt;&gt;&gt; sql = &quot;SELECT y.a AS a FROM (SELECT x.a AS a, x.b AS b FROM x) AS y&quot;</span> </span><span id="pushdown_projections-25"><a href="#pushdown_projections-25"><span class="linenos"> 25</span></a><span class="sd"> &gt;&gt;&gt; sql = &quot;SELECT y.a AS a FROM (SELECT x.a AS a, x.b AS b FROM x) AS y&quot;</span>
</span><span id="pushdown_projections-26"><a href="#pushdown_projections-26"><span class="linenos">26</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(sql)</span> </span><span id="pushdown_projections-26"><a href="#pushdown_projections-26"><span class="linenos"> 26</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(sql)</span>
</span><span id="pushdown_projections-27"><a href="#pushdown_projections-27"><span class="linenos">27</span></a><span class="sd"> &gt;&gt;&gt; pushdown_projections(expression).sql()</span> </span><span id="pushdown_projections-27"><a href="#pushdown_projections-27"><span class="linenos"> 27</span></a><span class="sd"> &gt;&gt;&gt; pushdown_projections(expression).sql()</span>
</span><span id="pushdown_projections-28"><a href="#pushdown_projections-28"><span class="linenos">28</span></a><span class="sd"> &#39;SELECT y.a AS a FROM (SELECT x.a AS a FROM x) AS y&#39;</span> </span><span id="pushdown_projections-28"><a href="#pushdown_projections-28"><span class="linenos"> 28</span></a><span class="sd"> &#39;SELECT y.a AS a FROM (SELECT x.a AS a FROM x) AS y&#39;</span>
</span><span id="pushdown_projections-29"><a href="#pushdown_projections-29"><span class="linenos">29</span></a> </span><span id="pushdown_projections-29"><a href="#pushdown_projections-29"><span class="linenos"> 29</span></a>
</span><span id="pushdown_projections-30"><a href="#pushdown_projections-30"><span class="linenos">30</span></a><span class="sd"> Args:</span> </span><span id="pushdown_projections-30"><a href="#pushdown_projections-30"><span class="linenos"> 30</span></a><span class="sd"> Args:</span>
</span><span id="pushdown_projections-31"><a href="#pushdown_projections-31"><span class="linenos">31</span></a><span class="sd"> expression (sqlglot.Expression): expression to optimize</span> </span><span id="pushdown_projections-31"><a href="#pushdown_projections-31"><span class="linenos"> 31</span></a><span class="sd"> expression (sqlglot.Expression): expression to optimize</span>
</span><span id="pushdown_projections-32"><a href="#pushdown_projections-32"><span class="linenos">32</span></a><span class="sd"> remove_unused_selections (bool): remove selects that are unused</span> </span><span id="pushdown_projections-32"><a href="#pushdown_projections-32"><span class="linenos"> 32</span></a><span class="sd"> remove_unused_selections (bool): remove selects that are unused</span>
</span><span id="pushdown_projections-33"><a href="#pushdown_projections-33"><span class="linenos">33</span></a><span class="sd"> Returns:</span> </span><span id="pushdown_projections-33"><a href="#pushdown_projections-33"><span class="linenos"> 33</span></a><span class="sd"> Returns:</span>
</span><span id="pushdown_projections-34"><a href="#pushdown_projections-34"><span class="linenos">34</span></a><span class="sd"> sqlglot.Expression: optimized expression</span> </span><span id="pushdown_projections-34"><a href="#pushdown_projections-34"><span class="linenos"> 34</span></a><span class="sd"> sqlglot.Expression: optimized expression</span>
</span><span id="pushdown_projections-35"><a href="#pushdown_projections-35"><span class="linenos">35</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="pushdown_projections-35"><a href="#pushdown_projections-35"><span class="linenos"> 35</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="pushdown_projections-36"><a href="#pushdown_projections-36"><span class="linenos">36</span></a> <span class="c1"># Map of Scope to all columns being selected by outer queries.</span> </span><span id="pushdown_projections-36"><a href="#pushdown_projections-36"><span class="linenos"> 36</span></a> <span class="c1"># Map of Scope to all columns being selected by outer queries.</span>
</span><span id="pushdown_projections-37"><a href="#pushdown_projections-37"><span class="linenos">37</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">)</span> </span><span id="pushdown_projections-37"><a href="#pushdown_projections-37"><span class="linenos"> 37</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">)</span>
</span><span id="pushdown_projections-38"><a href="#pushdown_projections-38"><span class="linenos">38</span></a> <span class="n">source_column_alias_count</span> <span class="o">=</span> <span class="p">{}</span> </span><span id="pushdown_projections-38"><a href="#pushdown_projections-38"><span class="linenos"> 38</span></a> <span class="n">source_column_alias_count</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="pushdown_projections-39"><a href="#pushdown_projections-39"><span class="linenos">39</span></a> <span class="n">referenced_columns</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">set</span><span class="p">)</span> </span><span id="pushdown_projections-39"><a href="#pushdown_projections-39"><span class="linenos"> 39</span></a> <span class="n">referenced_columns</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">set</span><span class="p">)</span>
</span><span id="pushdown_projections-40"><a href="#pushdown_projections-40"><span class="linenos">40</span></a> </span><span id="pushdown_projections-40"><a href="#pushdown_projections-40"><span class="linenos"> 40</span></a>
</span><span id="pushdown_projections-41"><a href="#pushdown_projections-41"><span class="linenos">41</span></a> <span class="c1"># We build the scope tree (which is traversed in DFS postorder), then iterate</span> </span><span id="pushdown_projections-41"><a href="#pushdown_projections-41"><span class="linenos"> 41</span></a> <span class="c1"># We build the scope tree (which is traversed in DFS postorder), then iterate</span>
</span><span id="pushdown_projections-42"><a href="#pushdown_projections-42"><span class="linenos">42</span></a> <span class="c1"># over the result in reverse order. This should ensure that the set of selected</span> </span><span id="pushdown_projections-42"><a href="#pushdown_projections-42"><span class="linenos"> 42</span></a> <span class="c1"># over the result in reverse order. This should ensure that the set of selected</span>
</span><span id="pushdown_projections-43"><a href="#pushdown_projections-43"><span class="linenos">43</span></a> <span class="c1"># columns for a particular scope are completely build by the time we get to it.</span> </span><span id="pushdown_projections-43"><a href="#pushdown_projections-43"><span class="linenos"> 43</span></a> <span class="c1"># columns for a particular scope are completely build by the time we get to it.</span>
</span><span id="pushdown_projections-44"><a href="#pushdown_projections-44"><span class="linenos">44</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="nb">reversed</span><span class="p">(</span><span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">)):</span> </span><span id="pushdown_projections-44"><a href="#pushdown_projections-44"><span class="linenos"> 44</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="nb">reversed</span><span class="p">(</span><span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">)):</span>
</span><span id="pushdown_projections-45"><a href="#pushdown_projections-45"><span class="linenos">45</span></a> <span class="n">parent_selections</span> <span class="o">=</span> <span class="n">referenced_columns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="p">{</span><span class="n">SELECT_ALL</span><span class="p">})</span> </span><span id="pushdown_projections-45"><a href="#pushdown_projections-45"><span class="linenos"> 45</span></a> <span class="n">parent_selections</span> <span class="o">=</span> <span class="n">referenced_columns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="p">{</span><span class="n">SELECT_ALL</span><span class="p">})</span>
</span><span id="pushdown_projections-46"><a href="#pushdown_projections-46"><span class="linenos">46</span></a> <span class="n">alias_count</span> <span class="o">=</span> <span class="n">source_column_alias_count</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> </span><span id="pushdown_projections-46"><a href="#pushdown_projections-46"><span class="linenos"> 46</span></a> <span class="n">alias_count</span> <span class="o">=</span> <span class="n">source_column_alias_count</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
</span><span id="pushdown_projections-47"><a href="#pushdown_projections-47"><span class="linenos">47</span></a> </span><span id="pushdown_projections-47"><a href="#pushdown_projections-47"><span class="linenos"> 47</span></a>
</span><span id="pushdown_projections-48"><a href="#pushdown_projections-48"><span class="linenos">48</span></a> <span class="c1"># We can&#39;t remove columns SELECT DISTINCT nor UNION DISTINCT.</span> </span><span id="pushdown_projections-48"><a href="#pushdown_projections-48"><span class="linenos"> 48</span></a> <span class="c1"># We can&#39;t remove columns SELECT DISTINCT nor UNION DISTINCT.</span>
</span><span id="pushdown_projections-49"><a href="#pushdown_projections-49"><span class="linenos">49</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;distinct&quot;</span><span class="p">):</span> </span><span id="pushdown_projections-49"><a href="#pushdown_projections-49"><span class="linenos"> 49</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;distinct&quot;</span><span class="p">):</span>
</span><span id="pushdown_projections-50"><a href="#pushdown_projections-50"><span class="linenos">50</span></a> <span class="n">parent_selections</span> <span class="o">=</span> <span class="p">{</span><span class="n">SELECT_ALL</span><span class="p">}</span> </span><span id="pushdown_projections-50"><a href="#pushdown_projections-50"><span class="linenos"> 50</span></a> <span class="n">parent_selections</span> <span class="o">=</span> <span class="p">{</span><span class="n">SELECT_ALL</span><span class="p">}</span>
</span><span id="pushdown_projections-51"><a href="#pushdown_projections-51"><span class="linenos">51</span></a> </span><span id="pushdown_projections-51"><a href="#pushdown_projections-51"><span class="linenos"> 51</span></a>
</span><span id="pushdown_projections-52"><a href="#pushdown_projections-52"><span class="linenos">52</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">SetOperation</span><span class="p">):</span> </span><span id="pushdown_projections-52"><a href="#pushdown_projections-52"><span class="linenos"> 52</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">SetOperation</span><span class="p">):</span>
</span><span id="pushdown_projections-53"><a href="#pushdown_projections-53"><span class="linenos">53</span></a> <span class="n">left</span><span class="p">,</span> <span class="n">right</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">union_scopes</span> </span><span id="pushdown_projections-53"><a href="#pushdown_projections-53"><span class="linenos"> 53</span></a> <span class="n">set_op</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span>
</span><span id="pushdown_projections-54"><a href="#pushdown_projections-54"><span class="linenos">54</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span> <span class="o">!=</span> <span class="nb">len</span><span class="p">(</span><span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span> </span><span id="pushdown_projections-54"><a href="#pushdown_projections-54"><span class="linenos"> 54</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="p">(</span><span class="n">set_op</span><span class="o">.</span><span class="n">kind</span> <span class="ow">or</span> <span class="n">set_op</span><span class="o">.</span><span class="n">side</span><span class="p">):</span>
</span><span id="pushdown_projections-55"><a href="#pushdown_projections-55"><span class="linenos">55</span></a> <span class="n">scope_sql</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">()</span> </span><span id="pushdown_projections-55"><a href="#pushdown_projections-55"><span class="linenos"> 55</span></a> <span class="c1"># Do not optimize this set operation if it&#39;s using the BigQuery specific</span>
</span><span id="pushdown_projections-56"><a href="#pushdown_projections-56"><span class="linenos">56</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;Invalid set operation due to column mismatch: </span><span class="si">{</span><span class="n">scope_sql</span><span class="si">}</span><span class="s2">.&quot;</span><span class="p">)</span> </span><span id="pushdown_projections-56"><a href="#pushdown_projections-56"><span class="linenos"> 56</span></a> <span class="c1"># kind / side syntax (e.g INNER UNION ALL BY NAME) which changes the semantics of the operation</span>
</span><span id="pushdown_projections-57"><a href="#pushdown_projections-57"><span class="linenos">57</span></a> </span><span id="pushdown_projections-57"><a href="#pushdown_projections-57"><span class="linenos"> 57</span></a> <span class="n">left</span><span class="p">,</span> <span class="n">right</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">union_scopes</span>
</span><span id="pushdown_projections-58"><a href="#pushdown_projections-58"><span class="linenos">58</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">left</span><span class="p">]</span> <span class="o">=</span> <span class="n">parent_selections</span> </span><span id="pushdown_projections-58"><a href="#pushdown_projections-58"><span class="linenos"> 58</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span> <span class="o">!=</span> <span class="nb">len</span><span class="p">(</span><span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
</span><span id="pushdown_projections-59"><a href="#pushdown_projections-59"><span class="linenos">59</span></a> </span><span id="pushdown_projections-59"><a href="#pushdown_projections-59"><span class="linenos"> 59</span></a> <span class="n">scope_sql</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">()</span>
</span><span id="pushdown_projections-60"><a href="#pushdown_projections-60"><span class="linenos">60</span></a> <span class="k">if</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span> </span><span id="pushdown_projections-60"><a href="#pushdown_projections-60"><span class="linenos"> 60</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span>
</span><span id="pushdown_projections-61"><a href="#pushdown_projections-61"><span class="linenos">61</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="n">parent_selections</span> </span><span id="pushdown_projections-61"><a href="#pushdown_projections-61"><span class="linenos"> 61</span></a> <span class="sa">f</span><span class="s2">&quot;Invalid set operation due to column mismatch: </span><span class="si">{</span><span class="n">scope_sql</span><span class="si">}</span><span class="s2">.&quot;</span>
</span><span id="pushdown_projections-62"><a href="#pushdown_projections-62"><span class="linenos">62</span></a> <span class="k">elif</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span> </span><span id="pushdown_projections-62"><a href="#pushdown_projections-62"><span class="linenos"> 62</span></a> <span class="p">)</span>
</span><span id="pushdown_projections-63"><a href="#pushdown_projections-63"><span class="linenos">63</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;by_name&quot;</span><span class="p">):</span> </span><span id="pushdown_projections-63"><a href="#pushdown_projections-63"><span class="linenos"> 63</span></a>
</span><span id="pushdown_projections-64"><a href="#pushdown_projections-64"><span class="linenos">64</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">left</span><span class="p">]</span> </span><span id="pushdown_projections-64"><a href="#pushdown_projections-64"><span class="linenos"> 64</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">left</span><span class="p">]</span> <span class="o">=</span> <span class="n">parent_selections</span>
</span><span id="pushdown_projections-65"><a href="#pushdown_projections-65"><span class="linenos">65</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="pushdown_projections-65"><a href="#pushdown_projections-65"><span class="linenos"> 65</span></a>
</span><span id="pushdown_projections-66"><a href="#pushdown_projections-66"><span class="linenos">66</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span> </span><span id="pushdown_projections-66"><a href="#pushdown_projections-66"><span class="linenos"> 66</span></a> <span class="k">if</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
</span><span id="pushdown_projections-67"><a href="#pushdown_projections-67"><span class="linenos">67</span></a> <span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">alias_or_name</span> </span><span id="pushdown_projections-67"><a href="#pushdown_projections-67"><span class="linenos"> 67</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="n">parent_selections</span>
</span><span id="pushdown_projections-68"><a href="#pushdown_projections-68"><span class="linenos">68</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">select</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span> </span><span id="pushdown_projections-68"><a href="#pushdown_projections-68"><span class="linenos"> 68</span></a> <span class="k">elif</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
</span><span id="pushdown_projections-69"><a href="#pushdown_projections-69"><span class="linenos">69</span></a> <span class="k">if</span> <span class="n">SELECT_ALL</span> <span class="ow">in</span> <span class="n">parent_selections</span> </span><span id="pushdown_projections-69"><a href="#pushdown_projections-69"><span class="linenos"> 69</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;by_name&quot;</span><span class="p">):</span>
</span><span id="pushdown_projections-70"><a href="#pushdown_projections-70"><span class="linenos">70</span></a> <span class="ow">or</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">in</span> <span class="n">parent_selections</span> </span><span id="pushdown_projections-70"><a href="#pushdown_projections-70"><span class="linenos"> 70</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">left</span><span class="p">]</span>
</span><span id="pushdown_projections-71"><a href="#pushdown_projections-71"><span class="linenos">71</span></a> <span class="p">]</span> </span><span id="pushdown_projections-71"><a href="#pushdown_projections-71"><span class="linenos"> 71</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="pushdown_projections-72"><a href="#pushdown_projections-72"><span class="linenos">72</span></a> </span><span id="pushdown_projections-72"><a href="#pushdown_projections-72"><span class="linenos"> 72</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span>
</span><span id="pushdown_projections-73"><a href="#pushdown_projections-73"><span class="linenos">73</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span> </span><span id="pushdown_projections-73"><a href="#pushdown_projections-73"><span class="linenos"> 73</span></a> <span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">alias_or_name</span>
</span><span id="pushdown_projections-74"><a href="#pushdown_projections-74"><span class="linenos">74</span></a> <span class="k">if</span> <span class="n">remove_unused_selections</span><span class="p">:</span> </span><span id="pushdown_projections-74"><a href="#pushdown_projections-74"><span class="linenos"> 74</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">select</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span>
</span><span id="pushdown_projections-75"><a href="#pushdown_projections-75"><span class="linenos">75</span></a> <span class="n">_remove_unused_selections</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">parent_selections</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">alias_count</span><span class="p">)</span> </span><span id="pushdown_projections-75"><a href="#pushdown_projections-75"><span class="linenos"> 75</span></a> <span class="k">if</span> <span class="n">SELECT_ALL</span> <span class="ow">in</span> <span class="n">parent_selections</span>
</span><span id="pushdown_projections-76"><a href="#pushdown_projections-76"><span class="linenos">76</span></a> </span><span id="pushdown_projections-76"><a href="#pushdown_projections-76"><span class="linenos"> 76</span></a> <span class="ow">or</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">in</span> <span class="n">parent_selections</span>
</span><span id="pushdown_projections-77"><a href="#pushdown_projections-77"><span class="linenos">77</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">is_star</span><span class="p">:</span> </span><span id="pushdown_projections-77"><a href="#pushdown_projections-77"><span class="linenos"> 77</span></a> <span class="p">]</span>
</span><span id="pushdown_projections-78"><a href="#pushdown_projections-78"><span class="linenos">78</span></a> <span class="k">continue</span> </span><span id="pushdown_projections-78"><a href="#pushdown_projections-78"><span class="linenos"> 78</span></a>
</span><span id="pushdown_projections-79"><a href="#pushdown_projections-79"><span class="linenos">79</span></a> </span><span id="pushdown_projections-79"><a href="#pushdown_projections-79"><span class="linenos"> 79</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="pushdown_projections-80"><a href="#pushdown_projections-80"><span class="linenos">80</span></a> <span class="c1"># Group columns by source name</span> </span><span id="pushdown_projections-80"><a href="#pushdown_projections-80"><span class="linenos"> 80</span></a> <span class="k">if</span> <span class="n">remove_unused_selections</span><span class="p">:</span>
</span><span id="pushdown_projections-81"><a href="#pushdown_projections-81"><span class="linenos">81</span></a> <span class="n">selects</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">set</span><span class="p">)</span> </span><span id="pushdown_projections-81"><a href="#pushdown_projections-81"><span class="linenos"> 81</span></a> <span class="n">_remove_unused_selections</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">parent_selections</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">alias_count</span><span class="p">)</span>
</span><span id="pushdown_projections-82"><a href="#pushdown_projections-82"><span class="linenos">82</span></a> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span> </span><span id="pushdown_projections-82"><a href="#pushdown_projections-82"><span class="linenos"> 82</span></a>
</span><span id="pushdown_projections-83"><a href="#pushdown_projections-83"><span class="linenos">83</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">table</span> </span><span id="pushdown_projections-83"><a href="#pushdown_projections-83"><span class="linenos"> 83</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">is_star</span><span class="p">:</span>
</span><span id="pushdown_projections-84"><a href="#pushdown_projections-84"><span class="linenos">84</span></a> <span class="n">col_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">name</span> </span><span id="pushdown_projections-84"><a href="#pushdown_projections-84"><span class="linenos"> 84</span></a> <span class="k">continue</span>
</span><span id="pushdown_projections-85"><a href="#pushdown_projections-85"><span class="linenos">85</span></a> <span class="n">selects</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span> </span><span id="pushdown_projections-85"><a href="#pushdown_projections-85"><span class="linenos"> 85</span></a>
</span><span id="pushdown_projections-86"><a href="#pushdown_projections-86"><span class="linenos">86</span></a> </span><span id="pushdown_projections-86"><a href="#pushdown_projections-86"><span class="linenos"> 86</span></a> <span class="c1"># Group columns by source name</span>
</span><span id="pushdown_projections-87"><a href="#pushdown_projections-87"><span class="linenos">87</span></a> <span class="c1"># Push the selected columns down to the next scope</span> </span><span id="pushdown_projections-87"><a href="#pushdown_projections-87"><span class="linenos"> 87</span></a> <span class="n">selects</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">set</span><span class="p">)</span>
</span><span id="pushdown_projections-88"><a href="#pushdown_projections-88"><span class="linenos">88</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">source</span><span class="p">)</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span> </span><span id="pushdown_projections-88"><a href="#pushdown_projections-88"><span class="linenos"> 88</span></a> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="pushdown_projections-89"><a href="#pushdown_projections-89"><span class="linenos">89</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span> </span><span id="pushdown_projections-89"><a href="#pushdown_projections-89"><span class="linenos"> 89</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">table</span>
</span><span id="pushdown_projections-90"><a href="#pushdown_projections-90"><span class="linenos">90</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">SELECT_ALL</span><span class="p">}</span> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">pivots</span> <span class="k">else</span> <span class="n">selects</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="ow">or</span> <span class="nb">set</span><span class="p">()</span> </span><span id="pushdown_projections-90"><a href="#pushdown_projections-90"><span class="linenos"> 90</span></a> <span class="n">col_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">name</span>
</span><span id="pushdown_projections-91"><a href="#pushdown_projections-91"><span class="linenos">91</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">source</span><span class="p">]</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span> </span><span id="pushdown_projections-91"><a href="#pushdown_projections-91"><span class="linenos"> 91</span></a> <span class="n">selects</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span>
</span><span id="pushdown_projections-92"><a href="#pushdown_projections-92"><span class="linenos">92</span></a> </span><span id="pushdown_projections-92"><a href="#pushdown_projections-92"><span class="linenos"> 92</span></a>
</span><span id="pushdown_projections-93"><a href="#pushdown_projections-93"><span class="linenos">93</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">alias_column_names</span> </span><span id="pushdown_projections-93"><a href="#pushdown_projections-93"><span class="linenos"> 93</span></a> <span class="c1"># Push the selected columns down to the next scope</span>
</span><span id="pushdown_projections-94"><a href="#pushdown_projections-94"><span class="linenos">94</span></a> <span class="k">if</span> <span class="n">column_aliases</span><span class="p">:</span> </span><span id="pushdown_projections-94"><a href="#pushdown_projections-94"><span class="linenos"> 94</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">source</span><span class="p">)</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="pushdown_projections-95"><a href="#pushdown_projections-95"><span class="linenos">95</span></a> <span class="n">source_column_alias_count</span><span class="p">[</span><span class="n">source</span><span class="p">]</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">column_aliases</span><span class="p">)</span> </span><span id="pushdown_projections-95"><a href="#pushdown_projections-95"><span class="linenos"> 95</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span>
</span><span id="pushdown_projections-96"><a href="#pushdown_projections-96"><span class="linenos">96</span></a> </span><span id="pushdown_projections-96"><a href="#pushdown_projections-96"><span class="linenos"> 96</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">SELECT_ALL</span><span class="p">}</span> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">pivots</span> <span class="k">else</span> <span class="n">selects</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="ow">or</span> <span class="nb">set</span><span class="p">()</span>
</span><span id="pushdown_projections-97"><a href="#pushdown_projections-97"><span class="linenos">97</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="pushdown_projections-97"><a href="#pushdown_projections-97"><span class="linenos"> 97</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">source</span><span class="p">]</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
</span><span id="pushdown_projections-98"><a href="#pushdown_projections-98"><span class="linenos"> 98</span></a>
</span><span id="pushdown_projections-99"><a href="#pushdown_projections-99"><span class="linenos"> 99</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">alias_column_names</span>
</span><span id="pushdown_projections-100"><a href="#pushdown_projections-100"><span class="linenos">100</span></a> <span class="k">if</span> <span class="n">column_aliases</span><span class="p">:</span>
</span><span id="pushdown_projections-101"><a href="#pushdown_projections-101"><span class="linenos">101</span></a> <span class="n">source_column_alias_count</span><span class="p">[</span><span class="n">source</span><span class="p">]</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">column_aliases</span><span class="p">)</span>
</span><span id="pushdown_projections-102"><a href="#pushdown_projections-102"><span class="linenos">102</span></a>
</span><span id="pushdown_projections-103"><a href="#pushdown_projections-103"><span class="linenos">103</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>

2496
docs/sqlglot/optimizer/qualify_columns.html
View file

File diff suppressed because it is too large Load diff

547
docs/sqlglot/optimizer/qualify_tables.html
View file

@ -66,147 +66,149 @@
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.helper</span><span class="w"> </span><span class="kn">import</span> <span class="n">csv_reader</span><span class="p">,</span> <span class="n">name_sequence</span> </span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.helper</span><span class="w"> </span><span class="kn">import</span> <span class="n">csv_reader</span><span class="p">,</span> <span class="n">name_sequence</span>
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.optimizer.scope</span><span class="w"> </span><span class="kn">import</span> <span class="n">Scope</span><span class="p">,</span> <span class="n">traverse_scope</span> </span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.optimizer.scope</span><span class="w"> </span><span class="kn">import</span> <span class="n">Scope</span><span class="p">,</span> <span class="n">traverse_scope</span>
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.schema</span><span class="w"> </span><span class="kn">import</span> <span class="n">Schema</span> </span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.schema</span><span class="w"> </span><span class="kn">import</span> <span class="n">Schema</span>
</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a> </span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.dialects.dialect</span><span class="w"> </span><span class="kn">import</span> <span class="n">Dialect</span>
</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a><span class="k">if</span> <span class="n">t</span><span class="o">.</span><span class="n">TYPE_CHECKING</span><span class="p">:</span> </span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a>
</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a> <span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot._typing</span><span class="w"> </span><span class="kn">import</span> <span class="n">E</span> </span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a><span class="k">if</span> <span class="n">t</span><span class="o">.</span><span class="n">TYPE_CHECKING</span><span class="p">:</span>
</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a> </span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a> <span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot._typing</span><span class="w"> </span><span class="kn">import</span> <span class="n">E</span>
</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a> </span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a>
</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="k">def</span><span class="w"> </span><span class="nf">qualify_tables</span><span class="p">(</span> </span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a>
</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span> </span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a><span class="k">def</span><span class="w"> </span><span class="nf">qualify_tables</span><span class="p">(</span>
</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span>
</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span> </span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span> </span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span>
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span> </span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span> </span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span>
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a> </span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span>
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a><span class="sd"> Examples:</span> </span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a>
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span> </span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="sd"> Examples:</span>
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM tbl&quot;)</span> </span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression, db=&quot;db&quot;).sql()</span> </span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM tbl&quot;)</span>
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a><span class="sd"> &#39;SELECT 1 FROM db.tbl AS tbl&#39;</span> </span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression, db=&quot;db&quot;).sql()</span>
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a><span class="sd"> &gt;&gt;&gt;</span> </span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a><span class="sd"> &#39;SELECT 1 FROM db.tbl AS tbl&#39;</span>
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM (t1 JOIN t2) AS t&quot;)</span> </span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a><span class="sd"> &gt;&gt;&gt;</span>
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression).sql()</span> </span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM (t1 JOIN t2) AS t&quot;)</span>
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a><span class="sd"> &#39;SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t&#39;</span> </span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression).sql()</span>
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a> </span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a><span class="sd"> &#39;SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t&#39;</span>
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a><span class="sd"> Args:</span> </span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a>
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a><span class="sd"> expression: Expression to qualify</span> </span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a><span class="sd"> Args:</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="sd"> db: Database name</span> </span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="sd"> expression: Expression to qualify</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a><span class="sd"> catalog: Catalog name</span> </span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a><span class="sd"> db: Database name</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a><span class="sd"> schema: A schema to populate</span> </span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a><span class="sd"> catalog: Catalog name</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a><span class="sd"> infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs&#39; schemas.</span> </span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a><span class="sd"> schema: A schema to populate</span>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a><span class="sd"> dialect: The dialect to parse catalog and schema into.</span> </span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a><span class="sd"> infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs&#39; schemas.</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> </span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a><span class="sd"> dialect: The dialect to parse catalog and schema into.</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a><span class="sd"> Returns:</span> </span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a><span class="sd"> The qualified expression.</span> </span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a><span class="sd"> Returns:</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a><span class="sd"> The qualified expression.</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">&quot;_q_&quot;</span><span class="p">)</span> </span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="n">db</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">db</span> <span class="k">else</span> <span class="kc">None</span> </span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">&quot;_q_&quot;</span><span class="p">)</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">catalog</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">catalog</span> <span class="k">else</span> <span class="kc">None</span> </span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">db</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">db</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> </span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="n">catalog</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">catalog</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">_qualify</span><span class="p">(</span><span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span> </span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span> </span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span> </span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">_qualify</span><span class="p">(</span><span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">,</span> <span class="n">db</span><span class="p">)</span> </span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">)</span> <span class="ow">and</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span> </span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">,</span> <span class="n">catalog</span><span class="p">)</span> </span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">,</span> <span class="n">db</span><span class="p">)</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> </span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">)</span> <span class="ow">and</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="k">if</span> <span class="p">(</span><span class="n">db</span> <span class="ow">or</span> <span class="n">catalog</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span> </span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">,</span> <span class="n">catalog</span><span class="p">)</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">)):</span> </span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span> </span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">if</span> <span class="p">(</span><span class="n">db</span> <span class="ow">or</span> <span class="n">catalog</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">node</span><span class="p">)</span> </span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">)):</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> </span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span> </span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span> </span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span> </span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span> </span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span> </span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span> </span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">&quot;*&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span> </span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span> </span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> </span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">&quot;*&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">):</span> </span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span> </span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span> </span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">):</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span> </span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> </span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span> </span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span> </span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span> </span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> </span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="n">table_aliases</span> <span class="o">=</span> <span class="p">{}</span> </span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> </span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span> </span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="n">table_aliases</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span> </span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span> </span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span> </span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="c1"># Don&#39;t add the pivot&#39;s alias to the pivoted table, use the table&#39;s name instead</span> </span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span> <span class="o">==</span> <span class="n">name</span><span class="p">:</span> </span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> </span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="c1"># Don&#39;t add the pivot&#39;s alias to the pivoted table, use the table&#39;s name instead</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> </span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span> <span class="o">==</span> <span class="n">name</span><span class="p">:</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span> </span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> </span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> </span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">table_aliases</span><span class="p">[</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">parts</span><span class="p">)]</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span> </span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">source</span><span class="o">.</span><span class="n">alias</span> </span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="p">)</span> </span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">table_aliases</span><span class="p">[</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">parts</span><span class="p">)]</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> </span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">source</span><span class="o">.</span><span class="n">alias</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">if</span> <span class="n">pivots</span><span class="p">:</span> </span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="p">)</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span> </span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="n">pivot_alias</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span> </span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">if</span> <span class="n">pivots</span><span class="p">:</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">pivot_alias</span><span class="p">)))</span> </span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> </span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="n">pivot_alias</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="c1"># This case corresponds to a pivoted CTE, we don&#39;t want to qualify that</span> </span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">pivot_alias</span><span class="p">)))</span>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">),</span> <span class="n">Scope</span><span class="p">):</span> </span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">continue</span> </span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="c1"># This case corresponds to a pivoted CTE, we don&#39;t want to qualify that</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> </span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">),</span> <span class="n">Scope</span><span class="p">):</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span> </span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="k">continue</span>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> </span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">if</span> <span class="n">infer_csv_schemas</span> <span class="ow">and</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span> </span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span> </span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span> </span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="k">if</span> <span class="n">infer_csv_schemas</span> <span class="ow">and</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span> </span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span> </span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">source</span><span class="p">,</span> </span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span> </span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> </span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">source</span><span class="p">,</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="p">)</span> </span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span> </span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span> </span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="p">)</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span> </span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span> </span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="p">)</span> </span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span> </span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> </span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="p">)</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> </span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span> </span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span> </span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span> </span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">))</span> </span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="n">dialect</span><span class="o">.</span><span class="n">generate_values_aliases</span><span class="p">(</span><span class="n">udtf</span><span class="p">)</span>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span> </span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">column_aliases</span><span class="p">)</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="k">if</span> <span class="p">(</span> </span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> </span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span> </span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span> </span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="p">):</span> </span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span> </span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> </span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="p">):</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> </span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span> </span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span> </span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span> </span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> </span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span> </span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span> </span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span> </span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;table&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span> </span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> </span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;table&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a>
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>
@ -222,142 +224,143 @@
</div> </div>
<a class="headerlink" href="#qualify_tables"></a> <a class="headerlink" href="#qualify_tables"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="qualify_tables-17"><a href="#qualify_tables-17"><span class="linenos"> 17</span></a><span class="k">def</span><span class="w"> </span><span class="nf">qualify_tables</span><span class="p">(</span> <div class="pdoc-code codehilite"><pre><span></span><span id="qualify_tables-18"><a href="#qualify_tables-18"><span class="linenos"> 18</span></a><span class="k">def</span><span class="w"> </span><span class="nf">qualify_tables</span><span class="p">(</span>
</span><span id="qualify_tables-18"><a href="#qualify_tables-18"><span class="linenos"> 18</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span> </span><span id="qualify_tables-19"><a href="#qualify_tables-19"><span class="linenos"> 19</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span>
</span><span id="qualify_tables-19"><a href="#qualify_tables-19"><span class="linenos"> 19</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="qualify_tables-20"><a href="#qualify_tables-20"><span class="linenos"> 20</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify_tables-20"><a href="#qualify_tables-20"><span class="linenos"> 20</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="qualify_tables-21"><a href="#qualify_tables-21"><span class="linenos"> 21</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify_tables-21"><a href="#qualify_tables-21"><span class="linenos"> 21</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="qualify_tables-22"><a href="#qualify_tables-22"><span class="linenos"> 22</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify_tables-22"><a href="#qualify_tables-22"><span class="linenos"> 22</span></a> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span> </span><span id="qualify_tables-23"><a href="#qualify_tables-23"><span class="linenos"> 23</span></a> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="qualify_tables-23"><a href="#qualify_tables-23"><span class="linenos"> 23</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="qualify_tables-24"><a href="#qualify_tables-24"><span class="linenos"> 24</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify_tables-24"><a href="#qualify_tables-24"><span class="linenos"> 24</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span> </span><span id="qualify_tables-25"><a href="#qualify_tables-25"><span class="linenos"> 25</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span>
</span><span id="qualify_tables-25"><a href="#qualify_tables-25"><span class="linenos"> 25</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="qualify_tables-26"><a href="#qualify_tables-26"><span class="linenos"> 26</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="qualify_tables-26"><a href="#qualify_tables-26"><span class="linenos"> 26</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span> </span><span id="qualify_tables-27"><a href="#qualify_tables-27"><span class="linenos"> 27</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span>
</span><span id="qualify_tables-27"><a href="#qualify_tables-27"><span class="linenos"> 27</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span> </span><span id="qualify_tables-28"><a href="#qualify_tables-28"><span class="linenos"> 28</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span>
</span><span id="qualify_tables-28"><a href="#qualify_tables-28"><span class="linenos"> 28</span></a> </span><span id="qualify_tables-29"><a href="#qualify_tables-29"><span class="linenos"> 29</span></a>
</span><span id="qualify_tables-29"><a href="#qualify_tables-29"><span class="linenos"> 29</span></a><span class="sd"> Examples:</span> </span><span id="qualify_tables-30"><a href="#qualify_tables-30"><span class="linenos"> 30</span></a><span class="sd"> Examples:</span>
</span><span id="qualify_tables-30"><a href="#qualify_tables-30"><span class="linenos"> 30</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span> </span><span id="qualify_tables-31"><a href="#qualify_tables-31"><span class="linenos"> 31</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="qualify_tables-31"><a href="#qualify_tables-31"><span class="linenos"> 31</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM tbl&quot;)</span> </span><span id="qualify_tables-32"><a href="#qualify_tables-32"><span class="linenos"> 32</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM tbl&quot;)</span>
</span><span id="qualify_tables-32"><a href="#qualify_tables-32"><span class="linenos"> 32</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression, db=&quot;db&quot;).sql()</span> </span><span id="qualify_tables-33"><a href="#qualify_tables-33"><span class="linenos"> 33</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression, db=&quot;db&quot;).sql()</span>
</span><span id="qualify_tables-33"><a href="#qualify_tables-33"><span class="linenos"> 33</span></a><span class="sd"> &#39;SELECT 1 FROM db.tbl AS tbl&#39;</span> </span><span id="qualify_tables-34"><a href="#qualify_tables-34"><span class="linenos"> 34</span></a><span class="sd"> &#39;SELECT 1 FROM db.tbl AS tbl&#39;</span>
</span><span id="qualify_tables-34"><a href="#qualify_tables-34"><span class="linenos"> 34</span></a><span class="sd"> &gt;&gt;&gt;</span> </span><span id="qualify_tables-35"><a href="#qualify_tables-35"><span class="linenos"> 35</span></a><span class="sd"> &gt;&gt;&gt;</span>
</span><span id="qualify_tables-35"><a href="#qualify_tables-35"><span class="linenos"> 35</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM (t1 JOIN t2) AS t&quot;)</span> </span><span id="qualify_tables-36"><a href="#qualify_tables-36"><span class="linenos"> 36</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM (t1 JOIN t2) AS t&quot;)</span>
</span><span id="qualify_tables-36"><a href="#qualify_tables-36"><span class="linenos"> 36</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression).sql()</span> </span><span id="qualify_tables-37"><a href="#qualify_tables-37"><span class="linenos"> 37</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression).sql()</span>
</span><span id="qualify_tables-37"><a href="#qualify_tables-37"><span class="linenos"> 37</span></a><span class="sd"> &#39;SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t&#39;</span> </span><span id="qualify_tables-38"><a href="#qualify_tables-38"><span class="linenos"> 38</span></a><span class="sd"> &#39;SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t&#39;</span>
</span><span id="qualify_tables-38"><a href="#qualify_tables-38"><span class="linenos"> 38</span></a> </span><span id="qualify_tables-39"><a href="#qualify_tables-39"><span class="linenos"> 39</span></a>
</span><span id="qualify_tables-39"><a href="#qualify_tables-39"><span class="linenos"> 39</span></a><span class="sd"> Args:</span> </span><span id="qualify_tables-40"><a href="#qualify_tables-40"><span class="linenos"> 40</span></a><span class="sd"> Args:</span>
</span><span id="qualify_tables-40"><a href="#qualify_tables-40"><span class="linenos"> 40</span></a><span class="sd"> expression: Expression to qualify</span> </span><span id="qualify_tables-41"><a href="#qualify_tables-41"><span class="linenos"> 41</span></a><span class="sd"> expression: Expression to qualify</span>
</span><span id="qualify_tables-41"><a href="#qualify_tables-41"><span class="linenos"> 41</span></a><span class="sd"> db: Database name</span> </span><span id="qualify_tables-42"><a href="#qualify_tables-42"><span class="linenos"> 42</span></a><span class="sd"> db: Database name</span>
</span><span id="qualify_tables-42"><a href="#qualify_tables-42"><span class="linenos"> 42</span></a><span class="sd"> catalog: Catalog name</span> </span><span id="qualify_tables-43"><a href="#qualify_tables-43"><span class="linenos"> 43</span></a><span class="sd"> catalog: Catalog name</span>
</span><span id="qualify_tables-43"><a href="#qualify_tables-43"><span class="linenos"> 43</span></a><span class="sd"> schema: A schema to populate</span> </span><span id="qualify_tables-44"><a href="#qualify_tables-44"><span class="linenos"> 44</span></a><span class="sd"> schema: A schema to populate</span>
</span><span id="qualify_tables-44"><a href="#qualify_tables-44"><span class="linenos"> 44</span></a><span class="sd"> infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs&#39; schemas.</span> </span><span id="qualify_tables-45"><a href="#qualify_tables-45"><span class="linenos"> 45</span></a><span class="sd"> infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs&#39; schemas.</span>
</span><span id="qualify_tables-45"><a href="#qualify_tables-45"><span class="linenos"> 45</span></a><span class="sd"> dialect: The dialect to parse catalog and schema into.</span> </span><span id="qualify_tables-46"><a href="#qualify_tables-46"><span class="linenos"> 46</span></a><span class="sd"> dialect: The dialect to parse catalog and schema into.</span>
</span><span id="qualify_tables-46"><a href="#qualify_tables-46"><span class="linenos"> 46</span></a> </span><span id="qualify_tables-47"><a href="#qualify_tables-47"><span class="linenos"> 47</span></a>
</span><span id="qualify_tables-47"><a href="#qualify_tables-47"><span class="linenos"> 47</span></a><span class="sd"> Returns:</span> </span><span id="qualify_tables-48"><a href="#qualify_tables-48"><span class="linenos"> 48</span></a><span class="sd"> Returns:</span>
</span><span id="qualify_tables-48"><a href="#qualify_tables-48"><span class="linenos"> 48</span></a><span class="sd"> The qualified expression.</span> </span><span id="qualify_tables-49"><a href="#qualify_tables-49"><span class="linenos"> 49</span></a><span class="sd"> The qualified expression.</span>
</span><span id="qualify_tables-49"><a href="#qualify_tables-49"><span class="linenos"> 49</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="qualify_tables-50"><a href="#qualify_tables-50"><span class="linenos"> 50</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="qualify_tables-50"><a href="#qualify_tables-50"><span class="linenos"> 50</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">&quot;_q_&quot;</span><span class="p">)</span> </span><span id="qualify_tables-51"><a href="#qualify_tables-51"><span class="linenos"> 51</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">&quot;_q_&quot;</span><span class="p">)</span>
</span><span id="qualify_tables-51"><a href="#qualify_tables-51"><span class="linenos"> 51</span></a> <span class="n">db</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">db</span> <span class="k">else</span> <span class="kc">None</span> </span><span id="qualify_tables-52"><a href="#qualify_tables-52"><span class="linenos"> 52</span></a> <span class="n">db</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">db</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="qualify_tables-52"><a href="#qualify_tables-52"><span class="linenos"> 52</span></a> <span class="n">catalog</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">catalog</span> <span class="k">else</span> <span class="kc">None</span> </span><span id="qualify_tables-53"><a href="#qualify_tables-53"><span class="linenos"> 53</span></a> <span class="n">catalog</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">catalog</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="qualify_tables-53"><a href="#qualify_tables-53"><span class="linenos"> 53</span></a> </span><span id="qualify_tables-54"><a href="#qualify_tables-54"><span class="linenos"> 54</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="qualify_tables-54"><a href="#qualify_tables-54"><span class="linenos"> 54</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">_qualify</span><span class="p">(</span><span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span> </span><span id="qualify_tables-55"><a href="#qualify_tables-55"><span class="linenos"> 55</span></a>
</span><span id="qualify_tables-55"><a href="#qualify_tables-55"><span class="linenos"> 55</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span> </span><span id="qualify_tables-56"><a href="#qualify_tables-56"><span class="linenos"> 56</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">_qualify</span><span class="p">(</span><span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="qualify_tables-56"><a href="#qualify_tables-56"><span class="linenos"> 56</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span> </span><span id="qualify_tables-57"><a href="#qualify_tables-57"><span class="linenos"> 57</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
</span><span id="qualify_tables-57"><a href="#qualify_tables-57"><span class="linenos"> 57</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">,</span> <span class="n">db</span><span class="p">)</span> </span><span id="qualify_tables-58"><a href="#qualify_tables-58"><span class="linenos"> 58</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="qualify_tables-58"><a href="#qualify_tables-58"><span class="linenos"> 58</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">)</span> <span class="ow">and</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span> </span><span id="qualify_tables-59"><a href="#qualify_tables-59"><span class="linenos"> 59</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">,</span> <span class="n">db</span><span class="p">)</span>
</span><span id="qualify_tables-59"><a href="#qualify_tables-59"><span class="linenos"> 59</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">,</span> <span class="n">catalog</span><span class="p">)</span> </span><span id="qualify_tables-60"><a href="#qualify_tables-60"><span class="linenos"> 60</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">)</span> <span class="ow">and</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="qualify_tables-60"><a href="#qualify_tables-60"><span class="linenos"> 60</span></a> </span><span id="qualify_tables-61"><a href="#qualify_tables-61"><span class="linenos"> 61</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">,</span> <span class="n">catalog</span><span class="p">)</span>
</span><span id="qualify_tables-61"><a href="#qualify_tables-61"><span class="linenos"> 61</span></a> <span class="k">if</span> <span class="p">(</span><span class="n">db</span> <span class="ow">or</span> <span class="n">catalog</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span> </span><span id="qualify_tables-62"><a href="#qualify_tables-62"><span class="linenos"> 62</span></a>
</span><span id="qualify_tables-62"><a href="#qualify_tables-62"><span class="linenos"> 62</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">)):</span> </span><span id="qualify_tables-63"><a href="#qualify_tables-63"><span class="linenos"> 63</span></a> <span class="k">if</span> <span class="p">(</span><span class="n">db</span> <span class="ow">or</span> <span class="n">catalog</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span>
</span><span id="qualify_tables-63"><a href="#qualify_tables-63"><span class="linenos"> 63</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span> </span><span id="qualify_tables-64"><a href="#qualify_tables-64"><span class="linenos"> 64</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">)):</span>
</span><span id="qualify_tables-64"><a href="#qualify_tables-64"><span class="linenos"> 64</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">node</span><span class="p">)</span> </span><span id="qualify_tables-65"><a href="#qualify_tables-65"><span class="linenos"> 65</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="qualify_tables-65"><a href="#qualify_tables-65"><span class="linenos"> 65</span></a> </span><span id="qualify_tables-66"><a href="#qualify_tables-66"><span class="linenos"> 66</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="qualify_tables-66"><a href="#qualify_tables-66"><span class="linenos"> 66</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span> </span><span id="qualify_tables-67"><a href="#qualify_tables-67"><span class="linenos"> 67</span></a>
</span><span id="qualify_tables-67"><a href="#qualify_tables-67"><span class="linenos"> 67</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span> </span><span id="qualify_tables-68"><a href="#qualify_tables-68"><span class="linenos"> 68</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
</span><span id="qualify_tables-68"><a href="#qualify_tables-68"><span class="linenos"> 68</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span> </span><span id="qualify_tables-69"><a href="#qualify_tables-69"><span class="linenos"> 69</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
</span><span id="qualify_tables-69"><a href="#qualify_tables-69"><span class="linenos"> 69</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span> </span><span id="qualify_tables-70"><a href="#qualify_tables-70"><span class="linenos"> 70</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
</span><span id="qualify_tables-70"><a href="#qualify_tables-70"><span class="linenos"> 70</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span> </span><span id="qualify_tables-71"><a href="#qualify_tables-71"><span class="linenos"> 71</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
</span><span id="qualify_tables-71"><a href="#qualify_tables-71"><span class="linenos"> 71</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span> </span><span id="qualify_tables-72"><a href="#qualify_tables-72"><span class="linenos"> 72</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="qualify_tables-72"><a href="#qualify_tables-72"><span class="linenos"> 72</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">&quot;*&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span> </span><span id="qualify_tables-73"><a href="#qualify_tables-73"><span class="linenos"> 73</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="qualify_tables-73"><a href="#qualify_tables-73"><span class="linenos"> 73</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span> </span><span id="qualify_tables-74"><a href="#qualify_tables-74"><span class="linenos"> 74</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">&quot;*&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
</span><span id="qualify_tables-74"><a href="#qualify_tables-74"><span class="linenos"> 74</span></a> </span><span id="qualify_tables-75"><a href="#qualify_tables-75"><span class="linenos"> 75</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span>
</span><span id="qualify_tables-75"><a href="#qualify_tables-75"><span class="linenos"> 75</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">):</span> </span><span id="qualify_tables-76"><a href="#qualify_tables-76"><span class="linenos"> 76</span></a>
</span><span id="qualify_tables-76"><a href="#qualify_tables-76"><span class="linenos"> 76</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span> </span><span id="qualify_tables-77"><a href="#qualify_tables-77"><span class="linenos"> 77</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">):</span>
</span><span id="qualify_tables-77"><a href="#qualify_tables-77"><span class="linenos"> 77</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span> </span><span id="qualify_tables-78"><a href="#qualify_tables-78"><span class="linenos"> 78</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
</span><span id="qualify_tables-78"><a href="#qualify_tables-78"><span class="linenos"> 78</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span> </span><span id="qualify_tables-79"><a href="#qualify_tables-79"><span class="linenos"> 79</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
</span><span id="qualify_tables-79"><a href="#qualify_tables-79"><span class="linenos"> 79</span></a> </span><span id="qualify_tables-80"><a href="#qualify_tables-80"><span class="linenos"> 80</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
</span><span id="qualify_tables-80"><a href="#qualify_tables-80"><span class="linenos"> 80</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span> </span><span id="qualify_tables-81"><a href="#qualify_tables-81"><span class="linenos"> 81</span></a>
</span><span id="qualify_tables-81"><a href="#qualify_tables-81"><span class="linenos"> 81</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span> </span><span id="qualify_tables-82"><a href="#qualify_tables-82"><span class="linenos"> 82</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="qualify_tables-82"><a href="#qualify_tables-82"><span class="linenos"> 82</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span> </span><span id="qualify_tables-83"><a href="#qualify_tables-83"><span class="linenos"> 83</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="qualify_tables-83"><a href="#qualify_tables-83"><span class="linenos"> 83</span></a> </span><span id="qualify_tables-84"><a href="#qualify_tables-84"><span class="linenos"> 84</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
</span><span id="qualify_tables-84"><a href="#qualify_tables-84"><span class="linenos"> 84</span></a> <span class="n">table_aliases</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="qualify_tables-85"><a href="#qualify_tables-85"><span class="linenos"> 85</span></a> </span><span id="qualify_tables-85"><a href="#qualify_tables-85"><span class="linenos"> 85</span></a>
</span><span id="qualify_tables-86"><a href="#qualify_tables-86"><span class="linenos"> 86</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span> </span><span id="qualify_tables-86"><a href="#qualify_tables-86"><span class="linenos"> 86</span></a> <span class="n">table_aliases</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="qualify_tables-87"><a href="#qualify_tables-87"><span class="linenos"> 87</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span> </span><span id="qualify_tables-87"><a href="#qualify_tables-87"><span class="linenos"> 87</span></a>
</span><span id="qualify_tables-88"><a href="#qualify_tables-88"><span class="linenos"> 88</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span> </span><span id="qualify_tables-88"><a href="#qualify_tables-88"><span class="linenos"> 88</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="qualify_tables-89"><a href="#qualify_tables-89"><span class="linenos"> 89</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span> </span><span id="qualify_tables-89"><a href="#qualify_tables-89"><span class="linenos"> 89</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="qualify_tables-90"><a href="#qualify_tables-90"><span class="linenos"> 90</span></a> <span class="c1"># Don&#39;t add the pivot&#39;s alias to the pivoted table, use the table&#39;s name instead</span> </span><span id="qualify_tables-90"><a href="#qualify_tables-90"><span class="linenos"> 90</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="qualify_tables-91"><a href="#qualify_tables-91"><span class="linenos"> 91</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span> <span class="o">==</span> <span class="n">name</span><span class="p">:</span> </span><span id="qualify_tables-91"><a href="#qualify_tables-91"><span class="linenos"> 91</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="qualify_tables-92"><a href="#qualify_tables-92"><span class="linenos"> 92</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> </span><span id="qualify_tables-92"><a href="#qualify_tables-92"><span class="linenos"> 92</span></a> <span class="c1"># Don&#39;t add the pivot&#39;s alias to the pivoted table, use the table&#39;s name instead</span>
</span><span id="qualify_tables-93"><a href="#qualify_tables-93"><span class="linenos"> 93</span></a> </span><span id="qualify_tables-93"><a href="#qualify_tables-93"><span class="linenos"> 93</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span> <span class="o">==</span> <span class="n">name</span><span class="p">:</span>
</span><span id="qualify_tables-94"><a href="#qualify_tables-94"><span class="linenos"> 94</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span> </span><span id="qualify_tables-94"><a href="#qualify_tables-94"><span class="linenos"> 94</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span>
</span><span id="qualify_tables-95"><a href="#qualify_tables-95"><span class="linenos"> 95</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> </span><span id="qualify_tables-95"><a href="#qualify_tables-95"><span class="linenos"> 95</span></a>
</span><span id="qualify_tables-96"><a href="#qualify_tables-96"><span class="linenos"> 96</span></a> </span><span id="qualify_tables-96"><a href="#qualify_tables-96"><span class="linenos"> 96</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span>
</span><span id="qualify_tables-97"><a href="#qualify_tables-97"><span class="linenos"> 97</span></a> <span class="n">table_aliases</span><span class="p">[</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">parts</span><span class="p">)]</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span> </span><span id="qualify_tables-97"><a href="#qualify_tables-97"><span class="linenos"> 97</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="qualify_tables-98"><a href="#qualify_tables-98"><span class="linenos"> 98</span></a> <span class="n">source</span><span class="o">.</span><span class="n">alias</span> </span><span id="qualify_tables-98"><a href="#qualify_tables-98"><span class="linenos"> 98</span></a>
</span><span id="qualify_tables-99"><a href="#qualify_tables-99"><span class="linenos"> 99</span></a> <span class="p">)</span> </span><span id="qualify_tables-99"><a href="#qualify_tables-99"><span class="linenos"> 99</span></a> <span class="n">table_aliases</span><span class="p">[</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">parts</span><span class="p">)]</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span>
</span><span id="qualify_tables-100"><a href="#qualify_tables-100"><span class="linenos">100</span></a> </span><span id="qualify_tables-100"><a href="#qualify_tables-100"><span class="linenos">100</span></a> <span class="n">source</span><span class="o">.</span><span class="n">alias</span>
</span><span id="qualify_tables-101"><a href="#qualify_tables-101"><span class="linenos">101</span></a> <span class="k">if</span> <span class="n">pivots</span><span class="p">:</span> </span><span id="qualify_tables-101"><a href="#qualify_tables-101"><span class="linenos">101</span></a> <span class="p">)</span>
</span><span id="qualify_tables-102"><a href="#qualify_tables-102"><span class="linenos">102</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span> </span><span id="qualify_tables-102"><a href="#qualify_tables-102"><span class="linenos">102</span></a>
</span><span id="qualify_tables-103"><a href="#qualify_tables-103"><span class="linenos">103</span></a> <span class="n">pivot_alias</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span> </span><span id="qualify_tables-103"><a href="#qualify_tables-103"><span class="linenos">103</span></a> <span class="k">if</span> <span class="n">pivots</span><span class="p">:</span>
</span><span id="qualify_tables-104"><a href="#qualify_tables-104"><span class="linenos">104</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">pivot_alias</span><span class="p">)))</span> </span><span id="qualify_tables-104"><a href="#qualify_tables-104"><span class="linenos">104</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="qualify_tables-105"><a href="#qualify_tables-105"><span class="linenos">105</span></a> </span><span id="qualify_tables-105"><a href="#qualify_tables-105"><span class="linenos">105</span></a> <span class="n">pivot_alias</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
</span><span id="qualify_tables-106"><a href="#qualify_tables-106"><span class="linenos">106</span></a> <span class="c1"># This case corresponds to a pivoted CTE, we don&#39;t want to qualify that</span> </span><span id="qualify_tables-106"><a href="#qualify_tables-106"><span class="linenos">106</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">pivot_alias</span><span class="p">)))</span>
</span><span id="qualify_tables-107"><a href="#qualify_tables-107"><span class="linenos">107</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">),</span> <span class="n">Scope</span><span class="p">):</span> </span><span id="qualify_tables-107"><a href="#qualify_tables-107"><span class="linenos">107</span></a>
</span><span id="qualify_tables-108"><a href="#qualify_tables-108"><span class="linenos">108</span></a> <span class="k">continue</span> </span><span id="qualify_tables-108"><a href="#qualify_tables-108"><span class="linenos">108</span></a> <span class="c1"># This case corresponds to a pivoted CTE, we don&#39;t want to qualify that</span>
</span><span id="qualify_tables-109"><a href="#qualify_tables-109"><span class="linenos">109</span></a> </span><span id="qualify_tables-109"><a href="#qualify_tables-109"><span class="linenos">109</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">),</span> <span class="n">Scope</span><span class="p">):</span>
</span><span id="qualify_tables-110"><a href="#qualify_tables-110"><span class="linenos">110</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span> </span><span id="qualify_tables-110"><a href="#qualify_tables-110"><span class="linenos">110</span></a> <span class="k">continue</span>
</span><span id="qualify_tables-111"><a href="#qualify_tables-111"><span class="linenos">111</span></a> </span><span id="qualify_tables-111"><a href="#qualify_tables-111"><span class="linenos">111</span></a>
</span><span id="qualify_tables-112"><a href="#qualify_tables-112"><span class="linenos">112</span></a> <span class="k">if</span> <span class="n">infer_csv_schemas</span> <span class="ow">and</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span> </span><span id="qualify_tables-112"><a href="#qualify_tables-112"><span class="linenos">112</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
</span><span id="qualify_tables-113"><a href="#qualify_tables-113"><span class="linenos">113</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span> </span><span id="qualify_tables-113"><a href="#qualify_tables-113"><span class="linenos">113</span></a>
</span><span id="qualify_tables-114"><a href="#qualify_tables-114"><span class="linenos">114</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span> </span><span id="qualify_tables-114"><a href="#qualify_tables-114"><span class="linenos">114</span></a> <span class="k">if</span> <span class="n">infer_csv_schemas</span> <span class="ow">and</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
</span><span id="qualify_tables-115"><a href="#qualify_tables-115"><span class="linenos">115</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span> </span><span id="qualify_tables-115"><a href="#qualify_tables-115"><span class="linenos">115</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
</span><span id="qualify_tables-116"><a href="#qualify_tables-116"><span class="linenos">116</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span> </span><span id="qualify_tables-116"><a href="#qualify_tables-116"><span class="linenos">116</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="qualify_tables-117"><a href="#qualify_tables-117"><span class="linenos">117</span></a> <span class="n">source</span><span class="p">,</span> </span><span id="qualify_tables-117"><a href="#qualify_tables-117"><span class="linenos">117</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="qualify_tables-118"><a href="#qualify_tables-118"><span class="linenos">118</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span> </span><span id="qualify_tables-118"><a href="#qualify_tables-118"><span class="linenos">118</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
</span><span id="qualify_tables-119"><a href="#qualify_tables-119"><span class="linenos">119</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> </span><span id="qualify_tables-119"><a href="#qualify_tables-119"><span class="linenos">119</span></a> <span class="n">source</span><span class="p">,</span>
</span><span id="qualify_tables-120"><a href="#qualify_tables-120"><span class="linenos">120</span></a> <span class="p">)</span> </span><span id="qualify_tables-120"><a href="#qualify_tables-120"><span class="linenos">120</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
</span><span id="qualify_tables-121"><a href="#qualify_tables-121"><span class="linenos">121</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span> </span><span id="qualify_tables-121"><a href="#qualify_tables-121"><span class="linenos">121</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="qualify_tables-122"><a href="#qualify_tables-122"><span class="linenos">122</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span> </span><span id="qualify_tables-122"><a href="#qualify_tables-122"><span class="linenos">122</span></a> <span class="p">)</span>
</span><span id="qualify_tables-123"><a href="#qualify_tables-123"><span class="linenos">123</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span> </span><span id="qualify_tables-123"><a href="#qualify_tables-123"><span class="linenos">123</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
</span><span id="qualify_tables-124"><a href="#qualify_tables-124"><span class="linenos">124</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span> </span><span id="qualify_tables-124"><a href="#qualify_tables-124"><span class="linenos">124</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="qualify_tables-125"><a href="#qualify_tables-125"><span class="linenos">125</span></a> <span class="p">)</span> </span><span id="qualify_tables-125"><a href="#qualify_tables-125"><span class="linenos">125</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="qualify_tables-126"><a href="#qualify_tables-126"><span class="linenos">126</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span> </span><span id="qualify_tables-126"><a href="#qualify_tables-126"><span class="linenos">126</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
</span><span id="qualify_tables-127"><a href="#qualify_tables-127"><span class="linenos">127</span></a> </span><span id="qualify_tables-127"><a href="#qualify_tables-127"><span class="linenos">127</span></a> <span class="p">)</span>
</span><span id="qualify_tables-128"><a href="#qualify_tables-128"><span class="linenos">128</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> </span><span id="qualify_tables-128"><a href="#qualify_tables-128"><span class="linenos">128</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="qualify_tables-129"><a href="#qualify_tables-129"><span class="linenos">129</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span> </span><span id="qualify_tables-129"><a href="#qualify_tables-129"><span class="linenos">129</span></a>
</span><span id="qualify_tables-130"><a href="#qualify_tables-130"><span class="linenos">130</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span> </span><span id="qualify_tables-130"><a href="#qualify_tables-130"><span class="linenos">130</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
</span><span id="qualify_tables-131"><a href="#qualify_tables-131"><span class="linenos">131</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span> </span><span id="qualify_tables-131"><a href="#qualify_tables-131"><span class="linenos">131</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="qualify_tables-132"><a href="#qualify_tables-132"><span class="linenos">132</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">))</span> </span><span id="qualify_tables-132"><a href="#qualify_tables-132"><span class="linenos">132</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="qualify_tables-133"><a href="#qualify_tables-133"><span class="linenos">133</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="qualify_tables-133"><a href="#qualify_tables-133"><span class="linenos">133</span></a> <span class="n">column_aliases</span> <span class="o">=</span> <span class="n">dialect</span><span class="o">.</span><span class="n">generate_values_aliases</span><span class="p">(</span><span class="n">udtf</span><span class="p">)</span>
</span><span id="qualify_tables-134"><a href="#qualify_tables-134"><span class="linenos">134</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span> </span><span id="qualify_tables-134"><a href="#qualify_tables-134"><span class="linenos">134</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">column_aliases</span><span class="p">)</span>
</span><span id="qualify_tables-135"><a href="#qualify_tables-135"><span class="linenos">135</span></a> <span class="k">if</span> <span class="p">(</span> </span><span id="qualify_tables-135"><a href="#qualify_tables-135"><span class="linenos">135</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="qualify_tables-136"><a href="#qualify_tables-136"><span class="linenos">136</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> </span><span id="qualify_tables-136"><a href="#qualify_tables-136"><span class="linenos">136</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
</span><span id="qualify_tables-137"><a href="#qualify_tables-137"><span class="linenos">137</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span> </span><span id="qualify_tables-137"><a href="#qualify_tables-137"><span class="linenos">137</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="qualify_tables-138"><a href="#qualify_tables-138"><span class="linenos">138</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span> </span><span id="qualify_tables-138"><a href="#qualify_tables-138"><span class="linenos">138</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
</span><span id="qualify_tables-139"><a href="#qualify_tables-139"><span class="linenos">139</span></a> <span class="p">):</span> </span><span id="qualify_tables-139"><a href="#qualify_tables-139"><span class="linenos">139</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span>
</span><span id="qualify_tables-140"><a href="#qualify_tables-140"><span class="linenos">140</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span> </span><span id="qualify_tables-140"><a href="#qualify_tables-140"><span class="linenos">140</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
</span><span id="qualify_tables-141"><a href="#qualify_tables-141"><span class="linenos">141</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> </span><span id="qualify_tables-141"><a href="#qualify_tables-141"><span class="linenos">141</span></a> <span class="p">):</span>
</span><span id="qualify_tables-142"><a href="#qualify_tables-142"><span class="linenos">142</span></a> </span><span id="qualify_tables-142"><a href="#qualify_tables-142"><span class="linenos">142</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span>
</span><span id="qualify_tables-143"><a href="#qualify_tables-143"><span class="linenos">143</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span> </span><span id="qualify_tables-143"><a href="#qualify_tables-143"><span class="linenos">143</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="qualify_tables-144"><a href="#qualify_tables-144"><span class="linenos">144</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span> </span><span id="qualify_tables-144"><a href="#qualify_tables-144"><span class="linenos">144</span></a>
</span><span id="qualify_tables-145"><a href="#qualify_tables-145"><span class="linenos">145</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span> </span><span id="qualify_tables-145"><a href="#qualify_tables-145"><span class="linenos">145</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="qualify_tables-146"><a href="#qualify_tables-146"><span class="linenos">146</span></a> </span><span id="qualify_tables-146"><a href="#qualify_tables-146"><span class="linenos">146</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span>
</span><span id="qualify_tables-147"><a href="#qualify_tables-147"><span class="linenos">147</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span> </span><span id="qualify_tables-147"><a href="#qualify_tables-147"><span class="linenos">147</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
</span><span id="qualify_tables-148"><a href="#qualify_tables-148"><span class="linenos">148</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span> </span><span id="qualify_tables-148"><a href="#qualify_tables-148"><span class="linenos">148</span></a>
</span><span id="qualify_tables-149"><a href="#qualify_tables-149"><span class="linenos">149</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span> </span><span id="qualify_tables-149"><a href="#qualify_tables-149"><span class="linenos">149</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span>
</span><span id="qualify_tables-150"><a href="#qualify_tables-150"><span class="linenos">150</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;table&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span> </span><span id="qualify_tables-150"><a href="#qualify_tables-150"><span class="linenos">150</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
</span><span id="qualify_tables-151"><a href="#qualify_tables-151"><span class="linenos">151</span></a> </span><span id="qualify_tables-151"><a href="#qualify_tables-151"><span class="linenos">151</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="qualify_tables-152"><a href="#qualify_tables-152"><span class="linenos">152</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="qualify_tables-152"><a href="#qualify_tables-152"><span class="linenos">152</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;table&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="qualify_tables-153"><a href="#qualify_tables-153"><span class="linenos">153</span></a>
</span><span id="qualify_tables-154"><a href="#qualify_tables-154"><span class="linenos">154</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>

4
docs/sqlglot/optimizer/simplify.html
View file

@ -3250,7 +3250,7 @@ prefix are statically known.</p>
<div class="attr variable"> <div class="attr variable">
<span class="name">DATETRUNC_COMPARISONS</span> = <span class="name">DATETRUNC_COMPARISONS</span> =
<input id="DATETRUNC_COMPARISONS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="DATETRUNC_COMPARISONS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="../expressions.html#In">sqlglot.expressions.In</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;}</span> <label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#In">sqlglot.expressions.In</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;}</span>
</div> </div>
@ -3334,7 +3334,7 @@ prefix are statically known.</p>
<section id="JOINS"> <section id="JOINS">
<div class="attr variable"> <div class="attr variable">
<span class="name">JOINS</span> = <span class="name">JOINS</span> =
<span class="default_value">{(&#39;RIGHT&#39;, &#39;&#39;), (&#39;&#39;, &#39;INNER&#39;), (&#39;RIGHT&#39;, &#39;OUTER&#39;), (&#39;&#39;, &#39;&#39;)}</span> <span class="default_value">{(&#39;&#39;, &#39;&#39;), (&#39;RIGHT&#39;, &#39;&#39;), (&#39;RIGHT&#39;, &#39;OUTER&#39;), (&#39;&#39;, &#39;INNER&#39;)}</span>
</div> </div>

31745
docs/sqlglot/parser.html
View file

File diff suppressed because one or more lines are too long

2
docs/sqlglot/time.html
View file

File diff suppressed because one or more lines are too long

5570
docs/sqlglot/tokens.html
View file

File diff suppressed because it is too large Load diff

2
sqlglot/dialects/athena.py
View file

@ -15,7 +15,7 @@ def _generate_as_hive(expression: exp.Expression) -> bool:
if properties and properties.find(exp.ExternalProperty): if properties and properties.find(exp.ExternalProperty):
return True # CREATE EXTERNAL TABLE is Hive return True # CREATE EXTERNAL TABLE is Hive
if not isinstance(expression.expression, exp.Select): if not isinstance(expression.expression, exp.Query):
return True # any CREATE TABLE other than CREATE TABLE AS SELECT is Hive return True # any CREATE TABLE other than CREATE TABLE AS SELECT is Hive
else: else:
return expression.kind != "VIEW" # CREATE VIEW is never Hive but CREATE SCHEMA etc is return expression.kind != "VIEW" # CREATE VIEW is never Hive but CREATE SCHEMA etc is

4
sqlglot/dialects/bigquery.py
View file

@ -800,7 +800,7 @@ class BigQuery(Dialect):
if unnest_expr: if unnest_expr:
from sqlglot.optimizer.annotate_types import annotate_types from sqlglot.optimizer.annotate_types import annotate_types
unnest_expr = annotate_types(unnest_expr) unnest_expr = annotate_types(unnest_expr, dialect=self.dialect)
# Unnesting a nested array (i.e array of structs) explodes the top-level struct fields, # Unnesting a nested array (i.e array of structs) explodes the top-level struct fields,
# in contrast to other dialects such as DuckDB which flattens only the array by default # in contrast to other dialects such as DuckDB which flattens only the array by default
@ -1227,7 +1227,7 @@ class BigQuery(Dialect):
if arg.type is None: if arg.type is None:
from sqlglot.optimizer.annotate_types import annotate_types from sqlglot.optimizer.annotate_types import annotate_types
arg = annotate_types(arg) arg = annotate_types(arg, dialect=self.dialect)
if arg.type and arg.type.this in exp.DataType.TEXT_TYPES: if arg.type and arg.type.this in exp.DataType.TEXT_TYPES:
# BQ doesn't support bracket syntax with string values for structs # BQ doesn't support bracket syntax with string values for structs

62
sqlglot/dialects/clickhouse.py
View file

@ -207,6 +207,32 @@ class ClickHouse(Dialect):
exp.Union: None, exp.Union: None,
} }
def generate_values_aliases(self, expression: exp.Values) -> t.List[exp.Identifier]:
# Clickhouse allows VALUES to have an embedded structure e.g:
# VALUES('person String, place String', ('Noah', 'Paris'), ...)
# In this case, we don't want to qualify the columns
values = expression.expressions[0].expressions
structure = (
values[0]
if (len(values) > 1 and values[0].is_string and isinstance(values[1], exp.Tuple))
else None
)
if structure:
# Split each column definition into the column name e.g:
# 'person String, place String' -> ['person', 'place']
structure_coldefs = [coldef.strip() for coldef in structure.name.split(",")]
column_aliases = [
exp.to_identifier(coldef.split(" ")[0]) for coldef in structure_coldefs
]
else:
# Default column aliases in CH are "c1", "c2", etc.
column_aliases = [
exp.to_identifier(f"c{i + 1}") for i in range(len(values[0].expressions))
]
return column_aliases
class Tokenizer(tokens.Tokenizer): class Tokenizer(tokens.Tokenizer):
COMMENTS = ["--", "#", "#!", ("/*", "*/")] COMMENTS = ["--", "#", "#!", ("/*", "*/")]
IDENTIFIERS = ['"', "`"] IDENTIFIERS = ['"', "`"]
@ -811,7 +837,7 @@ class ClickHouse(Dialect):
def _parse_on_property(self) -> t.Optional[exp.Expression]: def _parse_on_property(self) -> t.Optional[exp.Expression]:
index = self._index index = self._index
if self._match_text_seq("CLUSTER"): if self._match_text_seq("CLUSTER"):
this = self._parse_id_var() this = self._parse_string() or self._parse_id_var()
if this: if this:
return self.expression(exp.OnCluster, this=this) return self.expression(exp.OnCluster, this=this)
else: else:
@ -905,6 +931,25 @@ class ClickHouse(Dialect):
this = exp.Apply(this=this, expression=self._parse_var(any_token=True)) this = exp.Apply(this=this, expression=self._parse_var(any_token=True))
return this return this
def _parse_value(self, values: bool = True) -> t.Optional[exp.Tuple]:
value = super()._parse_value(values=values)
if not value:
return None
# In Clickhouse "SELECT * FROM VALUES (1, 2, 3)" generates a table with a single column, in contrast
# to other dialects. For this case, we canonicalize the values into a tuple-of-tuples AST if it's not already one.
# In INSERT INTO statements the same clause actually references multiple columns (opposite semantics),
# but the final result is not altered by the extra parentheses.
# Note: Clickhouse allows VALUES([structure], value, ...) so the branch checks for the last expression
expressions = value.expressions
if values and not isinstance(expressions[-1], exp.Tuple):
value.set(
"expressions",
[self.expression(exp.Tuple, expressions=[expr]) for expr in expressions],
)
return value
class Generator(generator.Generator): class Generator(generator.Generator):
QUERY_HINTS = False QUERY_HINTS = False
STRUCT_DELIMITER = ("(", ")") STRUCT_DELIMITER = ("(", ")")
@ -919,8 +964,8 @@ class ClickHouse(Dialect):
TABLE_HINTS = False TABLE_HINTS = False
GROUPINGS_SEP = "" GROUPINGS_SEP = ""
SET_OP_MODIFIERS = False SET_OP_MODIFIERS = False
VALUES_AS_TABLE = False
ARRAY_SIZE_NAME = "LENGTH" ARRAY_SIZE_NAME = "LENGTH"
WRAP_DERIVED_VALUES = False
STRING_TYPE_MAPPING = { STRING_TYPE_MAPPING = {
exp.DataType.Type.BLOB: "String", exp.DataType.Type.BLOB: "String",
@ -1315,3 +1360,16 @@ class ClickHouse(Dialect):
return self.sql(expression, "this") return self.sql(expression, "this")
return super().not_sql(expression) return super().not_sql(expression)
def values_sql(self, expression: exp.Values, values_as_table: bool = True) -> str:
# If the VALUES clause contains tuples of expressions, we need to treat it
# as a table since Clickhouse will automatically alias it as such.
alias = expression.args.get("alias")
if alias and alias.args.get("columns") and expression.expressions:
values = expression.expressions[0].expressions
values_as_table = any(isinstance(value, exp.Tuple) for value in values)
else:
values_as_table = True
return super().values_sql(expression, values_as_table=values_as_table)

9
sqlglot/dialects/databricks.py
View file

@ -1,6 +1,5 @@
from __future__ import annotations from __future__ import annotations
import typing as t
from sqlglot import exp, transforms, jsonpath from sqlglot import exp, transforms, jsonpath
from sqlglot.dialects.dialect import ( from sqlglot.dialects.dialect import (
@ -13,13 +12,6 @@ from sqlglot.dialects.spark import Spark
from sqlglot.tokens import TokenType from sqlglot.tokens import TokenType
def _build_json_extract(args: t.List) -> exp.JSONExtract:
# Transform GET_JSON_OBJECT(expr, '$.<path>') -> expr:<path>
this = args[0]
path = args[1].name.lstrip("$.")
return exp.JSONExtract(this=this, expression=path)
def _jsonextract_sql( def _jsonextract_sql(
self: Databricks.Generator, expression: exp.JSONExtract | exp.JSONExtractScalar self: Databricks.Generator, expression: exp.JSONExtract | exp.JSONExtractScalar
) -> str: ) -> str:
@ -46,7 +38,6 @@ class Databricks(Spark):
"DATE_ADD": build_date_delta(exp.DateAdd), "DATE_ADD": build_date_delta(exp.DateAdd),
"DATEDIFF": build_date_delta(exp.DateDiff), "DATEDIFF": build_date_delta(exp.DateDiff),
"DATE_DIFF": build_date_delta(exp.DateDiff), "DATE_DIFF": build_date_delta(exp.DateDiff),
"GET_JSON_OBJECT": _build_json_extract,
"TO_DATE": build_formatted_time(exp.TsOrDsToDate, "databricks"), "TO_DATE": build_formatted_time(exp.TsOrDsToDate, "databricks"),
} }

12
sqlglot/dialects/dialect.py
View file

@ -1025,6 +1025,12 @@ class Dialect(metaclass=_Dialect):
def version(self) -> Version: def version(self) -> Version:
return Version(self.settings.get("version", None)) return Version(self.settings.get("version", None))
def generate_values_aliases(self, expression: exp.Values) -> t.List[exp.Identifier]:
return [
exp.to_identifier(f"_col_{i}")
for i, _ in enumerate(expression.expressions[0].expressions)
]
DialectType = t.Union[str, Dialect, t.Type[Dialect], None] DialectType = t.Union[str, Dialect, t.Type[Dialect], None]
@ -1297,7 +1303,9 @@ def no_timestamp_sql(self: Generator, expression: exp.Timestamp) -> str:
if not zone: if not zone:
from sqlglot.optimizer.annotate_types import annotate_types from sqlglot.optimizer.annotate_types import annotate_types
target_type = annotate_types(expression).type or exp.DataType.Type.TIMESTAMP target_type = (
annotate_types(expression, dialect=self.dialect).type or exp.DataType.Type.TIMESTAMP
)
return self.sql(exp.cast(expression.this, target_type)) return self.sql(exp.cast(expression.this, target_type))
if zone.name.lower() in TIMEZONES: if zone.name.lower() in TIMEZONES:
return self.sql( return self.sql(
@ -1864,7 +1872,7 @@ def build_timetostr_or_tochar(args: t.List, dialect: Dialect) -> exp.TimeToStr |
if this and not this.type: if this and not this.type:
from sqlglot.optimizer.annotate_types import annotate_types from sqlglot.optimizer.annotate_types import annotate_types
annotate_types(this) annotate_types(this, dialect=dialect)
if this.is_type(*exp.DataType.TEMPORAL_TYPES): if this.is_type(*exp.DataType.TEMPORAL_TYPES):
dialect_name = dialect.__class__.__name__.lower() dialect_name = dialect.__class__.__name__.lower()
return build_formatted_time(exp.TimeToStr, dialect_name, default=True)(args) return build_formatted_time(exp.TimeToStr, dialect_name, default=True)(args)

31
sqlglot/dialects/duckdb.py
View file

@ -156,6 +156,13 @@ def _build_make_timestamp(args: t.List) -> exp.Expression:
) )
def _show_parser(*args: t.Any, **kwargs: t.Any) -> t.Callable[[DuckDB.Parser], exp.Show]:
def _parse(self: DuckDB.Parser) -> exp.Show:
return self._parse_show_duckdb(*args, **kwargs)
return _parse
def _struct_sql(self: DuckDB.Generator, expression: exp.Struct) -> str: def _struct_sql(self: DuckDB.Generator, expression: exp.Struct) -> str:
args: t.List[str] = [] args: t.List[str] = []
@ -348,6 +355,8 @@ class DuckDB(Dialect):
"$": TokenType.PARAMETER, "$": TokenType.PARAMETER,
} }
COMMANDS = tokens.Tokenizer.COMMANDS - {TokenType.SHOW}
class Parser(parser.Parser): class Parser(parser.Parser):
BITWISE = { BITWISE = {
**parser.Parser.BITWISE, **parser.Parser.BITWISE,
@ -369,6 +378,11 @@ class DuckDB(Dialect):
FUNCTIONS_WITH_ALIASED_ARGS = {*parser.Parser.FUNCTIONS_WITH_ALIASED_ARGS, "STRUCT_PACK"} FUNCTIONS_WITH_ALIASED_ARGS = {*parser.Parser.FUNCTIONS_WITH_ALIASED_ARGS, "STRUCT_PACK"}
SHOW_PARSERS = {
"TABLES": _show_parser("TABLES"),
"ALL TABLES": _show_parser("ALL TABLES"),
}
FUNCTIONS = { FUNCTIONS = {
**parser.Parser.FUNCTIONS, **parser.Parser.FUNCTIONS,
"ARRAY_REVERSE_SORT": _build_sort_array_desc, "ARRAY_REVERSE_SORT": _build_sort_array_desc,
@ -467,6 +481,7 @@ class DuckDB(Dialect):
**parser.Parser.STATEMENT_PARSERS, **parser.Parser.STATEMENT_PARSERS,
TokenType.ATTACH: lambda self: self._parse_attach_detach(), TokenType.ATTACH: lambda self: self._parse_attach_detach(),
TokenType.DETACH: lambda self: self._parse_attach_detach(is_attach=False), TokenType.DETACH: lambda self: self._parse_attach_detach(is_attach=False),
TokenType.SHOW: lambda self: self._parse_show(),
} }
def _parse_expression(self) -> t.Optional[exp.Expression]: def _parse_expression(self) -> t.Optional[exp.Expression]:
@ -581,6 +596,9 @@ class DuckDB(Dialect):
else self.expression(exp.Detach, this=this, exists=exists) else self.expression(exp.Detach, this=this, exists=exists)
) )
def _parse_show_duckdb(self, this: str) -> exp.Show:
return self.expression(exp.Show, this=this)
class Generator(generator.Generator): class Generator(generator.Generator):
PARAMETER_TOKEN = "$" PARAMETER_TOKEN = "$"
NAMED_PLACEHOLDER_TOKEN = "$" NAMED_PLACEHOLDER_TOKEN = "$"
@ -861,6 +879,9 @@ class DuckDB(Dialect):
PROPERTIES_LOCATION[exp.TemporaryProperty] = exp.Properties.Location.POST_CREATE PROPERTIES_LOCATION[exp.TemporaryProperty] = exp.Properties.Location.POST_CREATE
PROPERTIES_LOCATION[exp.ReturnsProperty] = exp.Properties.Location.POST_ALIAS PROPERTIES_LOCATION[exp.ReturnsProperty] = exp.Properties.Location.POST_ALIAS
def show_sql(self, expression: exp.Show) -> str:
return f"SHOW {expression.name}"
def fromiso8601timestamp_sql(self, expression: exp.FromISO8601Timestamp) -> str: def fromiso8601timestamp_sql(self, expression: exp.FromISO8601Timestamp) -> str:
return self.sql(exp.cast(expression.this, exp.DataType.Type.TIMESTAMPTZ)) return self.sql(exp.cast(expression.this, exp.DataType.Type.TIMESTAMPTZ))
@ -987,7 +1008,7 @@ class DuckDB(Dialect):
if not this.type: if not this.type:
from sqlglot.optimizer.annotate_types import annotate_types from sqlglot.optimizer.annotate_types import annotate_types
this = annotate_types(this) this = annotate_types(this, dialect=self.dialect)
if this.is_type(exp.DataType.Type.MAP): if this.is_type(exp.DataType.Type.MAP):
bracket = f"({bracket})[1]" bracket = f"({bracket})[1]"
@ -1021,7 +1042,7 @@ class DuckDB(Dialect):
if not arg.type: if not arg.type:
from sqlglot.optimizer.annotate_types import annotate_types from sqlglot.optimizer.annotate_types import annotate_types
arg = annotate_types(arg) arg = annotate_types(arg, dialect=self.dialect)
if arg.is_type(*exp.DataType.TEXT_TYPES): if arg.is_type(*exp.DataType.TEXT_TYPES):
return self.func("LENGTH", arg) return self.func("LENGTH", arg)
@ -1032,10 +1053,10 @@ class DuckDB(Dialect):
case = ( case = (
exp.case(self.func("TYPEOF", arg)) exp.case(self.func("TYPEOF", arg))
.when(
"'VARCHAR'", exp.Anonymous(this="LENGTH", expressions=[varchar])
) # anonymous to break length_sql recursion
.when("'BLOB'", self.func("OCTET_LENGTH", blob)) .when("'BLOB'", self.func("OCTET_LENGTH", blob))
.else_(
exp.Anonymous(this="LENGTH", expressions=[varchar])
) # anonymous to break length_sql recursion
) )
return self.sql(case) return self.sql(case)

4
sqlglot/dialects/hive.py
View file

@ -307,7 +307,9 @@ class Hive(Dialect):
"FIRST": _build_with_ignore_nulls(exp.First), "FIRST": _build_with_ignore_nulls(exp.First),
"FIRST_VALUE": _build_with_ignore_nulls(exp.FirstValue), "FIRST_VALUE": _build_with_ignore_nulls(exp.FirstValue),
"FROM_UNIXTIME": build_formatted_time(exp.UnixToStr, "hive", True), "FROM_UNIXTIME": build_formatted_time(exp.UnixToStr, "hive", True),
"GET_JSON_OBJECT": exp.JSONExtractScalar.from_arg_list, "GET_JSON_OBJECT": lambda args, dialect: exp.JSONExtractScalar(
this=seq_get(args, 0), expression=dialect.to_json_path(seq_get(args, 1))
),
"LAST": _build_with_ignore_nulls(exp.Last), "LAST": _build_with_ignore_nulls(exp.Last),
"LAST_VALUE": _build_with_ignore_nulls(exp.LastValue), "LAST_VALUE": _build_with_ignore_nulls(exp.LastValue),
"MAP": parser.build_var_map, "MAP": parser.build_var_map,

9
sqlglot/dialects/postgres.py
View file

@ -41,6 +41,9 @@ from sqlglot.helper import is_int, seq_get
from sqlglot.parser import binary_range_parser from sqlglot.parser import binary_range_parser
from sqlglot.tokens import TokenType from sqlglot.tokens import TokenType
if t.TYPE_CHECKING:
from sqlglot.dialects.dialect import DialectType
DATE_DIFF_FACTOR = { DATE_DIFF_FACTOR = {
"MICROSECOND": " * 1000000", "MICROSECOND": " * 1000000",
@ -191,7 +194,7 @@ def _json_extract_sql(
return _generate return _generate
def _build_regexp_replace(args: t.List) -> exp.RegexpReplace: def _build_regexp_replace(args: t.List, dialect: DialectType = None) -> exp.RegexpReplace:
# The signature of REGEXP_REPLACE is: # The signature of REGEXP_REPLACE is:
# regexp_replace(source, pattern, replacement [, start [, N ]] [, flags ]) # regexp_replace(source, pattern, replacement [, start [, N ]] [, flags ])
# #
@ -204,7 +207,7 @@ def _build_regexp_replace(args: t.List) -> exp.RegexpReplace:
if not last.type or last.is_type(exp.DataType.Type.UNKNOWN, exp.DataType.Type.NULL): if not last.type or last.is_type(exp.DataType.Type.UNKNOWN, exp.DataType.Type.NULL):
from sqlglot.optimizer.annotate_types import annotate_types from sqlglot.optimizer.annotate_types import annotate_types
last = annotate_types(last) last = annotate_types(last, dialect=dialect)
if last.is_type(*exp.DataType.TEXT_TYPES): if last.is_type(*exp.DataType.TEXT_TYPES):
regexp_replace = exp.RegexpReplace.from_arg_list(args[:-1]) regexp_replace = exp.RegexpReplace.from_arg_list(args[:-1])
@ -657,7 +660,7 @@ class Postgres(Dialect):
from sqlglot.optimizer.annotate_types import annotate_types from sqlglot.optimizer.annotate_types import annotate_types
this = annotate_types(arg) this = annotate_types(arg, dialect=self.dialect)
if this.is_type("array<json>"): if this.is_type("array<json>"):
while isinstance(this, exp.Cast): while isinstance(this, exp.Cast):
this = this.this this = this.this

22
sqlglot/dialects/presto.py
View file

@ -332,6 +332,9 @@ class Presto(Dialect):
"FROM_UTF8": lambda args: exp.Decode( "FROM_UTF8": lambda args: exp.Decode(
this=seq_get(args, 0), replace=seq_get(args, 1), charset=exp.Literal.string("utf-8") this=seq_get(args, 0), replace=seq_get(args, 1), charset=exp.Literal.string("utf-8")
), ),
"JSON_FORMAT": lambda args: exp.JSONFormat(
this=seq_get(args, 0), options=seq_get(args, 1), is_json=True
),
"LEVENSHTEIN_DISTANCE": exp.Levenshtein.from_arg_list, "LEVENSHTEIN_DISTANCE": exp.Levenshtein.from_arg_list,
"NOW": exp.CurrentTimestamp.from_arg_list, "NOW": exp.CurrentTimestamp.from_arg_list,
"REGEXP_EXTRACT": build_regexp_extract(exp.RegexpExtract), "REGEXP_EXTRACT": build_regexp_extract(exp.RegexpExtract),
@ -582,13 +585,27 @@ class Presto(Dialect):
"with", "with",
} }
def jsonformat_sql(self, expression: exp.JSONFormat) -> str:
this = expression.this
is_json = expression.args.get("is_json")
if this and not (is_json or this.type):
from sqlglot.optimizer.annotate_types import annotate_types
this = annotate_types(this, dialect=self.dialect)
if not (is_json or this.is_type(exp.DataType.Type.JSON)):
this.replace(exp.cast(this, exp.DataType.Type.JSON))
return self.function_fallback_sql(expression)
def md5_sql(self, expression: exp.MD5) -> str: def md5_sql(self, expression: exp.MD5) -> str:
this = expression.this this = expression.this
if not this.type: if not this.type:
from sqlglot.optimizer.annotate_types import annotate_types from sqlglot.optimizer.annotate_types import annotate_types
this = annotate_types(this) this = annotate_types(this, dialect=self.dialect)
if this.is_type(*exp.DataType.TEXT_TYPES): if this.is_type(*exp.DataType.TEXT_TYPES):
this = exp.Encode(this=this, charset=exp.Literal.string("utf-8")) this = exp.Encode(this=this, charset=exp.Literal.string("utf-8"))
@ -630,6 +647,7 @@ class Presto(Dialect):
expression.this, expression.this,
expression.expressions, expression.expressions,
1 - expression.args.get("offset", 0), 1 - expression.args.get("offset", 0),
dialect=self.dialect,
), ),
0, 0,
), ),
@ -639,7 +657,7 @@ class Presto(Dialect):
def struct_sql(self, expression: exp.Struct) -> str: def struct_sql(self, expression: exp.Struct) -> str:
from sqlglot.optimizer.annotate_types import annotate_types from sqlglot.optimizer.annotate_types import annotate_types
expression = annotate_types(expression) expression = annotate_types(expression, dialect=self.dialect)
values: t.List[str] = [] values: t.List[str] = []
schema: t.List[str] = [] schema: t.List[str] = []
unknown_type = False unknown_type = False

40
sqlglot/dialects/snowflake.py
View file

@ -193,7 +193,7 @@ def _unqualify_pivot_columns(expression: exp.Expression) -> exp.Expression:
if expression.unpivot: if expression.unpivot:
expression = transforms.unqualify_columns(expression) expression = transforms.unqualify_columns(expression)
else: else:
field = expression.args.get("field") for field in expression.fields:
field_expr = seq_get(field.expressions if field else [], 0) field_expr = seq_get(field.expressions if field else [], 0)
if isinstance(field_expr, exp.PivotAny): if isinstance(field_expr, exp.PivotAny):
@ -516,6 +516,8 @@ class Snowflake(Dialect):
PROPERTY_PARSERS = { PROPERTY_PARSERS = {
**parser.Parser.PROPERTY_PARSERS, **parser.Parser.PROPERTY_PARSERS,
"CREDENTIALS": lambda self: self._parse_credentials_property(),
"FILE_FORMAT": lambda self: self._parse_file_format_property(),
"LOCATION": lambda self: self._parse_location_property(), "LOCATION": lambda self: self._parse_location_property(),
"TAG": lambda self: self._parse_tag(), "TAG": lambda self: self._parse_tag(),
"USING": lambda self: self._match_text_seq("TEMPLATE") "USING": lambda self: self._match_text_seq("TEMPLATE")
@ -546,9 +548,11 @@ class Snowflake(Dialect):
"TERSE UNIQUE KEYS": _show_parser("UNIQUE KEYS"), "TERSE UNIQUE KEYS": _show_parser("UNIQUE KEYS"),
"SEQUENCES": _show_parser("SEQUENCES"), "SEQUENCES": _show_parser("SEQUENCES"),
"TERSE SEQUENCES": _show_parser("SEQUENCES"), "TERSE SEQUENCES": _show_parser("SEQUENCES"),
"STAGES": _show_parser("STAGES"),
"COLUMNS": _show_parser("COLUMNS"), "COLUMNS": _show_parser("COLUMNS"),
"USERS": _show_parser("USERS"), "USERS": _show_parser("USERS"),
"TERSE USERS": _show_parser("USERS"), "TERSE USERS": _show_parser("USERS"),
"FILE FORMATS": _show_parser("FILE FORMATS"),
"FUNCTIONS": _show_parser("FUNCTIONS"), "FUNCTIONS": _show_parser("FUNCTIONS"),
"PROCEDURES": _show_parser("PROCEDURES"), "PROCEDURES": _show_parser("PROCEDURES"),
"WAREHOUSES": _show_parser("WAREHOUSES"), "WAREHOUSES": _show_parser("WAREHOUSES"),
@ -898,6 +902,24 @@ class Snowflake(Dialect):
# outoflineFK, explicitly names the columns # outoflineFK, explicitly names the columns
return super()._parse_foreign_key() return super()._parse_foreign_key()
def _parse_file_format_property(self) -> exp.FileFormatProperty:
self._match(TokenType.EQ)
if self._match(TokenType.L_PAREN, advance=False):
expressions = self._parse_wrapped_options()
else:
expressions = [self._parse_format_name()]
return self.expression(
exp.FileFormatProperty,
expressions=expressions,
)
def _parse_credentials_property(self) -> exp.CredentialsProperty:
return self.expression(
exp.CredentialsProperty,
expressions=self._parse_wrapped_options(),
)
class Tokenizer(tokens.Tokenizer): class Tokenizer(tokens.Tokenizer):
STRING_ESCAPES = ["\\", "'"] STRING_ESCAPES = ["\\", "'"]
HEX_STRINGS = [("x'", "'"), ("X'", "'")] HEX_STRINGS = [("x'", "'"), ("X'", "'")]
@ -912,6 +934,7 @@ class Snowflake(Dialect):
"CHAR VARYING": TokenType.VARCHAR, "CHAR VARYING": TokenType.VARCHAR,
"CHARACTER VARYING": TokenType.VARCHAR, "CHARACTER VARYING": TokenType.VARCHAR,
"EXCLUDE": TokenType.EXCEPT, "EXCLUDE": TokenType.EXCEPT,
"FILE FORMAT": TokenType.FILE_FORMAT,
"ILIKE ANY": TokenType.ILIKE_ANY, "ILIKE ANY": TokenType.ILIKE_ANY,
"LIKE ANY": TokenType.LIKE_ANY, "LIKE ANY": TokenType.LIKE_ANY,
"MATCH_CONDITION": TokenType.MATCH_CONDITION, "MATCH_CONDITION": TokenType.MATCH_CONDITION,
@ -929,6 +952,7 @@ class Snowflake(Dialect):
"TIMESTAMP_TZ": TokenType.TIMESTAMPTZ, "TIMESTAMP_TZ": TokenType.TIMESTAMPTZ,
"TOP": TokenType.TOP, "TOP": TokenType.TOP,
"WAREHOUSE": TokenType.WAREHOUSE, "WAREHOUSE": TokenType.WAREHOUSE,
"STAGE": TokenType.STAGE,
"STREAMLIT": TokenType.STREAMLIT, "STREAMLIT": TokenType.STREAMLIT,
} }
KEYWORDS.pop("/*+") KEYWORDS.pop("/*+")
@ -992,6 +1016,8 @@ class Snowflake(Dialect):
exp.DayOfYear: rename_func("DAYOFYEAR"), exp.DayOfYear: rename_func("DAYOFYEAR"),
exp.Explode: rename_func("FLATTEN"), exp.Explode: rename_func("FLATTEN"),
exp.Extract: rename_func("DATE_PART"), exp.Extract: rename_func("DATE_PART"),
exp.FileFormatProperty: lambda self,
e: f"FILE_FORMAT=({self.expressions(e, 'expressions', sep=' ')})",
exp.FromTimeZone: lambda self, e: self.func( exp.FromTimeZone: lambda self, e: self.func(
"CONVERT_TIMEZONE", e.args.get("zone"), "'UTC'", e.this "CONVERT_TIMEZONE", e.args.get("zone"), "'UTC'", e.this
), ),
@ -1007,6 +1033,10 @@ class Snowflake(Dialect):
exp.JSONObject: lambda self, e: self.func("OBJECT_CONSTRUCT_KEEP_NULL", *e.expressions), exp.JSONObject: lambda self, e: self.func("OBJECT_CONSTRUCT_KEEP_NULL", *e.expressions),
exp.JSONPathRoot: lambda *_: "", exp.JSONPathRoot: lambda *_: "",
exp.JSONValueArray: _json_extract_value_array_sql, exp.JSONValueArray: _json_extract_value_array_sql,
exp.Levenshtein: unsupported_args("ins_cost", "del_cost", "sub_cost")(
rename_func("EDITDISTANCE")
),
exp.LocationProperty: lambda self, e: f"LOCATION={self.sql(e, 'this')}",
exp.LogicalAnd: rename_func("BOOLAND_AGG"), exp.LogicalAnd: rename_func("BOOLAND_AGG"),
exp.LogicalOr: rename_func("BOOLOR_AGG"), exp.LogicalOr: rename_func("BOOLOR_AGG"),
exp.Map: lambda self, e: var_map_sql(self, e, "OBJECT_CONSTRUCT"), exp.Map: lambda self, e: var_map_sql(self, e, "OBJECT_CONSTRUCT"),
@ -1070,9 +1100,6 @@ class Snowflake(Dialect):
exp.VarMap: lambda self, e: var_map_sql(self, e, "OBJECT_CONSTRUCT"), exp.VarMap: lambda self, e: var_map_sql(self, e, "OBJECT_CONSTRUCT"),
exp.WeekOfYear: rename_func("WEEKOFYEAR"), exp.WeekOfYear: rename_func("WEEKOFYEAR"),
exp.Xor: rename_func("BOOLXOR"), exp.Xor: rename_func("BOOLXOR"),
exp.Levenshtein: unsupported_args("ins_cost", "del_cost", "sub_cost")(
rename_func("EDITDISTANCE")
),
} }
SUPPORTED_JSON_PATH_PARTS = { SUPPORTED_JSON_PATH_PARTS = {
@ -1093,6 +1120,9 @@ class Snowflake(Dialect):
PROPERTIES_LOCATION = { PROPERTIES_LOCATION = {
**generator.Generator.PROPERTIES_LOCATION, **generator.Generator.PROPERTIES_LOCATION,
exp.CredentialsProperty: exp.Properties.Location.POST_WITH,
exp.LocationProperty: exp.Properties.Location.POST_WITH,
exp.PartitionedByProperty: exp.Properties.Location.POST_SCHEMA,
exp.SetProperty: exp.Properties.Location.UNSUPPORTED, exp.SetProperty: exp.Properties.Location.UNSUPPORTED,
exp.VolatileProperty: exp.Properties.Location.UNSUPPORTED, exp.VolatileProperty: exp.Properties.Location.UNSUPPORTED,
} }
@ -1156,7 +1186,7 @@ class Snowflake(Dialect):
if value.type is None: if value.type is None:
from sqlglot.optimizer.annotate_types import annotate_types from sqlglot.optimizer.annotate_types import annotate_types
value = annotate_types(value) value = annotate_types(value, dialect=self.dialect)
if value.is_type(*exp.DataType.TEXT_TYPES, exp.DataType.Type.UNKNOWN): if value.is_type(*exp.DataType.TEXT_TYPES, exp.DataType.Type.UNKNOWN):
return super().trycast_sql(expression) return super().trycast_sql(expression)

6
sqlglot/dialects/spark2.py
View file

@ -104,7 +104,9 @@ def _unqualify_pivot_columns(expression: exp.Expression) -> exp.Expression:
SELECT * FROM tbl PIVOT(SUM(tbl.sales) FOR quarter IN ('Q1', 'Q1')) SELECT * FROM tbl PIVOT(SUM(tbl.sales) FOR quarter IN ('Q1', 'Q1'))
""" """
if isinstance(expression, exp.Pivot): if isinstance(expression, exp.Pivot):
expression.set("field", transforms.unqualify_columns(expression.args["field"])) expression.set(
"fields", [transforms.unqualify_columns(field) for field in expression.fields]
)
return expression return expression
@ -237,7 +239,7 @@ class Spark2(Hive):
def _pivot_column_names(self, aggregations: t.List[exp.Expression]) -> t.List[str]: def _pivot_column_names(self, aggregations: t.List[exp.Expression]) -> t.List[str]:
if len(aggregations) == 1: if len(aggregations) == 1:
return [""] return []
return pivot_column_names(aggregations, dialect="spark") return pivot_column_names(aggregations, dialect="spark")
class Generator(Hive.Generator): class Generator(Hive.Generator):

18
sqlglot/dialects/tsql.py
View file

@ -109,22 +109,25 @@ def _build_formatted_time(
exp_class: t.Type[E], full_format_mapping: t.Optional[bool] = None exp_class: t.Type[E], full_format_mapping: t.Optional[bool] = None
) -> t.Callable[[t.List], E]: ) -> t.Callable[[t.List], E]:
def _builder(args: t.List) -> E: def _builder(args: t.List) -> E:
assert len(args) == 2 fmt = seq_get(args, 0)
if isinstance(fmt, exp.Expression):
return exp_class( fmt = exp.Literal.string(
this=exp.cast(args[1], exp.DataType.Type.DATETIME2),
format=exp.Literal.string(
format_time( format_time(
args[0].name.lower(), fmt.name.lower(),
( (
{**TSQL.TIME_MAPPING, **FULL_FORMAT_TIME_MAPPING} {**TSQL.TIME_MAPPING, **FULL_FORMAT_TIME_MAPPING}
if full_format_mapping if full_format_mapping
else TSQL.TIME_MAPPING else TSQL.TIME_MAPPING
), ),
) )
),
) )
this = seq_get(args, 1)
if isinstance(this, exp.Expression):
this = exp.cast(this, exp.DataType.Type.DATETIME2)
return exp_class(this=this, format=fmt)
return _builder return _builder
@ -508,6 +511,7 @@ class TSQL(Dialect):
"DECLARE": TokenType.DECLARE, "DECLARE": TokenType.DECLARE,
"EXEC": TokenType.COMMAND, "EXEC": TokenType.COMMAND,
"FOR SYSTEM_TIME": TokenType.TIMESTAMP_SNAPSHOT, "FOR SYSTEM_TIME": TokenType.TIMESTAMP_SNAPSHOT,
"GO": TokenType.COMMAND,
"IMAGE": TokenType.IMAGE, "IMAGE": TokenType.IMAGE,
"MONEY": TokenType.MONEY, "MONEY": TokenType.MONEY,
"NONCLUSTERED INDEX": TokenType.INDEX, "NONCLUSTERED INDEX": TokenType.INDEX,

4
sqlglot/executor/__init__.py
View file

@ -65,7 +65,9 @@ def execute(
for column in table.columns: for column in table.columns:
value = table[0][column] value = table[0][column]
column_type = annotate_types(exp.convert(value)).type or type(value).__name__ column_type = (
annotate_types(exp.convert(value), dialect=read).type or type(value).__name__
)
nested_set(schema, [*keys, column], column_type) nested_set(schema, [*keys, column], column_type)
schema = ensure_schema(schema, dialect=read) schema = ensure_schema(schema, dialect=read)

31
sqlglot/expressions.py
View file

@ -2584,6 +2584,7 @@ class Lateral(UDTF):
"outer": False, "outer": False,
"alias": False, "alias": False,
"cross_apply": False, # True -> CROSS APPLY, False -> OUTER APPLY "cross_apply": False, # True -> CROSS APPLY, False -> OUTER APPLY
"ordinality": False,
} }
@ -2660,6 +2661,10 @@ class Sort(Order):
class Ordered(Expression): class Ordered(Expression):
arg_types = {"this": True, "desc": False, "nulls_first": True, "with_fill": False} arg_types = {"this": True, "desc": False, "nulls_first": True, "with_fill": False}
@property
def name(self) -> str:
return self.this.name
class Property(Expression): class Property(Expression):
arg_types = {"this": True, "value": True} arg_types = {"this": True, "value": True}
@ -2781,7 +2786,11 @@ class FallbackProperty(Property):
class FileFormatProperty(Property): class FileFormatProperty(Property):
arg_types = {"this": True} arg_types = {"this": False, "expressions": False}
class CredentialsProperty(Property):
arg_types = {"expressions": True}
class FreespaceProperty(Property): class FreespaceProperty(Property):
@ -3129,6 +3138,7 @@ class Properties(Expression):
"CLUSTERED_BY": ClusteredByProperty, "CLUSTERED_BY": ClusteredByProperty,
"COLLATE": CollateProperty, "COLLATE": CollateProperty,
"COMMENT": SchemaCommentProperty, "COMMENT": SchemaCommentProperty,
"CREDENTIALS": CredentialsProperty,
"DEFINER": DefinerProperty, "DEFINER": DefinerProperty,
"DISTKEY": DistKeyProperty, "DISTKEY": DistKeyProperty,
"DISTRIBUTED_BY": DistributedByProperty, "DISTRIBUTED_BY": DistributedByProperty,
@ -3366,6 +3376,9 @@ class SetOperation(Query):
"expression": True, "expression": True,
"distinct": False, "distinct": False,
"by_name": False, "by_name": False,
"side": False,
"kind": False,
"on": False,
**QUERY_MODIFIERS, **QUERY_MODIFIERS,
} }
@ -3404,6 +3417,14 @@ class SetOperation(Query):
def right(self) -> Query: def right(self) -> Query:
return self.expression return self.expression
@property
def kind(self) -> str:
return self.text("kind").upper()
@property
def side(self) -> str:
return self.text("side").upper()
class Union(SetOperation): class Union(SetOperation):
pass pass
@ -4307,7 +4328,7 @@ class Pivot(Expression):
"this": False, "this": False,
"alias": False, "alias": False,
"expressions": False, "expressions": False,
"field": False, "fields": False,
"unpivot": False, "unpivot": False,
"using": False, "using": False,
"group": False, "group": False,
@ -4321,6 +4342,10 @@ class Pivot(Expression):
def unpivot(self) -> bool: def unpivot(self) -> bool:
return bool(self.args.get("unpivot")) return bool(self.args.get("unpivot"))
@property
def fields(self) -> t.List[Expression]:
return self.args.get("fields", [])
# https://duckdb.org/docs/sql/statements/unpivot#simplified-unpivot-syntax # https://duckdb.org/docs/sql/statements/unpivot#simplified-unpivot-syntax
# UNPIVOT ... INTO [NAME <col_name> VALUE <col_value>][...,] # UNPIVOT ... INTO [NAME <col_name> VALUE <col_value>][...,]
@ -6286,7 +6311,7 @@ class JSONBExtractScalar(Binary, Func):
class JSONFormat(Func): class JSONFormat(Func):
arg_types = {"this": False, "options": False} arg_types = {"this": False, "options": False, "is_json": False}
_sql_names = ["JSON_FORMAT"] _sql_names = ["JSON_FORMAT"]

41
sqlglot/generator.py
View file

@ -132,6 +132,8 @@ class Generator(metaclass=_Generator):
exp.CommentColumnConstraint: lambda self, e: f"COMMENT {self.sql(e, 'this')}", exp.CommentColumnConstraint: lambda self, e: f"COMMENT {self.sql(e, 'this')}",
exp.ConnectByRoot: lambda self, e: f"CONNECT_BY_ROOT {self.sql(e, 'this')}", exp.ConnectByRoot: lambda self, e: f"CONNECT_BY_ROOT {self.sql(e, 'this')}",
exp.CopyGrantsProperty: lambda *_: "COPY GRANTS", exp.CopyGrantsProperty: lambda *_: "COPY GRANTS",
exp.CredentialsProperty: lambda self,
e: f"CREDENTIALS=({self.expressions(e, 'expressions', sep=' ')})",
exp.DateFormatColumnConstraint: lambda self, e: f"FORMAT {self.sql(e, 'this')}", exp.DateFormatColumnConstraint: lambda self, e: f"FORMAT {self.sql(e, 'this')}",
exp.DefaultColumnConstraint: lambda self, e: f"DEFAULT {self.sql(e, 'this')}", exp.DefaultColumnConstraint: lambda self, e: f"DEFAULT {self.sql(e, 'this')}",
exp.DynamicProperty: lambda *_: "DYNAMIC", exp.DynamicProperty: lambda *_: "DYNAMIC",
@ -1445,12 +1447,18 @@ class Generator(metaclass=_Generator):
self.unsupported(f"{op_name} requires DISTINCT or ALL to be specified") self.unsupported(f"{op_name} requires DISTINCT or ALL to be specified")
if distinct is default_distinct: if distinct is default_distinct:
kind = "" distinct_or_all = ""
else: else:
kind = " DISTINCT" if distinct else " ALL" distinct_or_all = " DISTINCT" if distinct else " ALL"
side_kind = " ".join(filter(None, [expression.side, expression.kind]))
side_kind = f"{side_kind} " if side_kind else ""
by_name = " BY NAME" if expression.args.get("by_name") else "" by_name = " BY NAME" if expression.args.get("by_name") else ""
return f"{op_name}{kind}{by_name}" on = self.expressions(expression, key="on", flat=True)
on = f" ON ({on})" if on else ""
return f"{side_kind}{op_name}{distinct_or_all}{by_name}{on}"
def set_operations(self, expression: exp.SetOperation) -> str: def set_operations(self, expression: exp.SetOperation) -> str:
if not self.SET_OP_MODIFIERS: if not self.SET_OP_MODIFIERS:
@ -2045,6 +2053,8 @@ class Generator(metaclass=_Generator):
expressions = self.expressions(expression, flat=True) expressions = self.expressions(expression, flat=True)
direction = "UNPIVOT" if expression.unpivot else "PIVOT" direction = "UNPIVOT" if expression.unpivot else "PIVOT"
group = self.sql(expression, "group")
if expression.this: if expression.this:
this = self.sql(expression, "this") this = self.sql(expression, "this")
if not expressions: if not expressions:
@ -2055,13 +2065,20 @@ class Generator(metaclass=_Generator):
into = f"{self.seg('INTO')} {into}" if into else "" into = f"{self.seg('INTO')} {into}" if into else ""
using = self.expressions(expression, key="using", flat=True) using = self.expressions(expression, key="using", flat=True)
using = f"{self.seg('USING')} {using}" if using else "" using = f"{self.seg('USING')} {using}" if using else ""
group = self.sql(expression, "group")
return f"{direction} {this}{on}{into}{using}{group}" return f"{direction} {this}{on}{into}{using}{group}"
alias = self.sql(expression, "alias") alias = self.sql(expression, "alias")
alias = f" AS {alias}" if alias else "" alias = f" AS {alias}" if alias else ""
field = self.sql(expression, "field") fields = self.expressions(
expression,
"fields",
sep=" ",
dynamic=True,
new_line=True,
skip_first=True,
skip_last=True,
)
include_nulls = expression.args.get("include_nulls") include_nulls = expression.args.get("include_nulls")
if include_nulls is not None: if include_nulls is not None:
@ -2071,7 +2088,7 @@ class Generator(metaclass=_Generator):
default_on_null = self.sql(expression, "default_on_null") default_on_null = self.sql(expression, "default_on_null")
default_on_null = f" DEFAULT ON NULL ({default_on_null})" if default_on_null else "" default_on_null = f" DEFAULT ON NULL ({default_on_null})" if default_on_null else ""
return f"{self.seg(direction)}{nulls}({expressions} FOR {field}{default_on_null}){alias}" return f"{self.seg(direction)}{nulls}({expressions} FOR {fields}{default_on_null}{group}){alias}"
def version_sql(self, expression: exp.Version) -> str: def version_sql(self, expression: exp.Version) -> str:
this = f"FOR {expression.name}" this = f"FOR {expression.name}"
@ -2292,7 +2309,13 @@ class Generator(metaclass=_Generator):
alias = self.sql(expression, "alias") alias = self.sql(expression, "alias")
alias = f" AS {alias}" if alias else "" alias = f" AS {alias}" if alias else ""
return f"{self.lateral_op(expression)} {this}{alias}"
ordinality = expression.args.get("ordinality") or ""
if ordinality:
ordinality = f" WITH ORDINALITY{alias}"
alias = ""
return f"{self.lateral_op(expression)} {this}{alias}{ordinality}"
def limit_sql(self, expression: exp.Limit, top: bool = False) -> str: def limit_sql(self, expression: exp.Limit, top: bool = False) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
@ -2558,6 +2581,7 @@ class Generator(metaclass=_Generator):
return f" {options}" if options else "" return f" {options}" if options else ""
def queryoption_sql(self, expression: exp.QueryOption) -> str: def queryoption_sql(self, expression: exp.QueryOption) -> str:
self.unsupported("Unsupported query option.")
return "" return ""
def offset_limit_modifiers( def offset_limit_modifiers(
@ -2786,6 +2810,7 @@ class Generator(metaclass=_Generator):
expression.this, expression.this,
expression.expressions, expression.expressions,
(index_offset or self.dialect.INDEX_OFFSET) - expression.args.get("offset", 0), (index_offset or self.dialect.INDEX_OFFSET) - expression.args.get("offset", 0),
dialect=self.dialect,
) )
def bracket_sql(self, expression: exp.Bracket) -> str: def bracket_sql(self, expression: exp.Bracket) -> str:
@ -3997,7 +4022,7 @@ class Generator(metaclass=_Generator):
if not arg.type: if not arg.type:
from sqlglot.optimizer.annotate_types import annotate_types from sqlglot.optimizer.annotate_types import annotate_types
arg = annotate_types(arg) arg = annotate_types(arg, dialect=self.dialect)
if arg.is_type(exp.DataType.Type.ARRAY): if arg.is_type(exp.DataType.Type.ARRAY):
return self.sql(arg) return self.sql(arg)

7
sqlglot/helper.py
View file

@ -15,6 +15,7 @@ from itertools import count
if t.TYPE_CHECKING: if t.TYPE_CHECKING:
from sqlglot import exp from sqlglot import exp
from sqlglot._typing import A, E, T from sqlglot._typing import A, E, T
from sqlglot.dialects.dialect import DialectType
from sqlglot.expressions import Expression from sqlglot.expressions import Expression
@ -150,6 +151,7 @@ def apply_index_offset(
this: exp.Expression, this: exp.Expression,
expressions: t.List[E], expressions: t.List[E],
offset: int, offset: int,
dialect: DialectType = None,
) -> t.List[E]: ) -> t.List[E]:
""" """
Applies an offset to a given integer literal expression. Applies an offset to a given integer literal expression.
@ -158,6 +160,7 @@ def apply_index_offset(
this: The target of the index. this: The target of the index.
expressions: The expression the offset will be applied to, wrapped in a list. expressions: The expression the offset will be applied to, wrapped in a list.
offset: The offset that will be applied. offset: The offset that will be applied.
dialect: the dialect of interest.
Returns: Returns:
The original expression with the offset applied to it, wrapped in a list. If the provided The original expression with the offset applied to it, wrapped in a list. If the provided
@ -173,7 +176,7 @@ def apply_index_offset(
from sqlglot.optimizer.simplify import simplify from sqlglot.optimizer.simplify import simplify
if not this.type: if not this.type:
annotate_types(this) annotate_types(this, dialect=dialect)
if t.cast(exp.DataType, this.type).this not in ( if t.cast(exp.DataType, this.type).this not in (
exp.DataType.Type.UNKNOWN, exp.DataType.Type.UNKNOWN,
@ -182,7 +185,7 @@ def apply_index_offset(
return expressions return expressions
if not expression.type: if not expression.type:
annotate_types(expression) annotate_types(expression, dialect=dialect)
if t.cast(exp.DataType, expression.type).this in exp.DataType.INTEGER_TYPES: if t.cast(exp.DataType, expression.type).this in exp.DataType.INTEGER_TYPES:
logger.info("Applying array index offset (%s)", offset) logger.info("Applying array index offset (%s)", offset)

6
sqlglot/optimizer/canonicalize.py
View file

@ -23,7 +23,7 @@ def canonicalize(expression: exp.Expression, dialect: DialectType = None) -> exp
def _canonicalize(expression: exp.Expression) -> exp.Expression: def _canonicalize(expression: exp.Expression) -> exp.Expression:
expression = add_text_to_concat(expression) expression = add_text_to_concat(expression)
expression = replace_date_funcs(expression) expression = replace_date_funcs(expression, dialect=dialect)
expression = coerce_type(expression, dialect.PROMOTE_TO_INFERRED_DATETIME_TYPE) expression = coerce_type(expression, dialect.PROMOTE_TO_INFERRED_DATETIME_TYPE)
expression = remove_redundant_casts(expression) expression = remove_redundant_casts(expression)
expression = ensure_bools(expression, _replace_int_predicate) expression = ensure_bools(expression, _replace_int_predicate)
@ -39,7 +39,7 @@ def add_text_to_concat(node: exp.Expression) -> exp.Expression:
return node return node
def replace_date_funcs(node: exp.Expression) -> exp.Expression: def replace_date_funcs(node: exp.Expression, dialect: DialectType) -> exp.Expression:
if ( if (
isinstance(node, (exp.Date, exp.TsOrDsToDate)) isinstance(node, (exp.Date, exp.TsOrDsToDate))
and not node.expressions and not node.expressions
@ -52,7 +52,7 @@ def replace_date_funcs(node: exp.Expression) -> exp.Expression:
if not node.type: if not node.type:
from sqlglot.optimizer.annotate_types import annotate_types from sqlglot.optimizer.annotate_types import annotate_types
node = annotate_types(node) node = annotate_types(node, dialect=dialect)
return exp.cast(node.this, to=node.type or exp.DataType.Type.TIMESTAMP) return exp.cast(node.this, to=node.type or exp.DataType.Type.TIMESTAMP)
return node return node

6
sqlglot/optimizer/merge_subqueries.py
View file

@ -237,12 +237,12 @@ def _rename_inner_sources(outer_scope: Scope, inner_scope: Scope, alias: str) ->
source, _ = inner_scope.selected_sources[conflict] source, _ = inner_scope.selected_sources[conflict]
new_alias = exp.to_identifier(new_name) new_alias = exp.to_identifier(new_name)
if isinstance(source, exp.Subquery): if isinstance(source, exp.Table) and source.alias:
source.set("alias", exp.TableAlias(this=new_alias))
elif isinstance(source, exp.Table) and source.alias:
source.set("alias", new_alias) source.set("alias", new_alias)
elif isinstance(source, exp.Table): elif isinstance(source, exp.Table):
source.replace(exp.alias_(source, new_alias)) source.replace(exp.alias_(source, new_alias))
elif isinstance(source.parent, exp.Subquery):
source.parent.set("alias", exp.TableAlias(this=new_alias))
for column in inner_scope.source_columns(conflict): for column in inner_scope.source_columns(conflict):
column.set("table", exp.to_identifier(new_name)) column.set("table", exp.to_identifier(new_name))

8
sqlglot/optimizer/pushdown_projections.py
View file

@ -49,10 +49,16 @@ def pushdown_projections(expression, schema=None, remove_unused_selections=True)
parent_selections = {SELECT_ALL} parent_selections = {SELECT_ALL}
if isinstance(scope.expression, exp.SetOperation): if isinstance(scope.expression, exp.SetOperation):
set_op = scope.expression
if not (set_op.kind or set_op.side):
# Do not optimize this set operation if it's using the BigQuery specific
# kind / side syntax (e.g INNER UNION ALL BY NAME) which changes the semantics of the operation
left, right = scope.union_scopes left, right = scope.union_scopes
if len(left.expression.selects) != len(right.expression.selects): if len(left.expression.selects) != len(right.expression.selects):
scope_sql = scope.expression.sql() scope_sql = scope.expression.sql()
raise OptimizeError(f"Invalid set operation due to column mismatch: {scope_sql}.") raise OptimizeError(
f"Invalid set operation due to column mismatch: {scope_sql}."
)
referenced_columns[left] = parent_selections referenced_columns[left] = parent_selections

48
sqlglot/optimizer/qualify_columns.py
View file

@ -140,13 +140,14 @@ def validate_qualify_columns(expression: E) -> E:
def _unpivot_columns(unpivot: exp.Pivot) -> t.Iterator[exp.Column]: def _unpivot_columns(unpivot: exp.Pivot) -> t.Iterator[exp.Column]:
name_column = [] name_columns = [
field = unpivot.args.get("field") field.this
if isinstance(field, exp.In) and isinstance(field.this, exp.Column): for field in unpivot.fields
name_column.append(field.this) if isinstance(field, exp.In) and isinstance(field.this, exp.Column)
]
value_columns = (c for e in unpivot.expressions for c in e.find_all(exp.Column)) value_columns = (c for e in unpivot.expressions for c in e.find_all(exp.Column))
return itertools.chain(name_column, value_columns)
return itertools.chain(name_columns, value_columns)
def _pop_table_column_aliases(derived_tables: t.List[exp.CTE | exp.Subquery]) -> None: def _pop_table_column_aliases(derived_tables: t.List[exp.CTE | exp.Subquery]) -> None:
@ -608,18 +609,19 @@ def _expand_stars(
dialect = resolver.schema.dialect dialect = resolver.schema.dialect
pivot_output_columns = None pivot_output_columns = None
pivot_exclude_columns = None pivot_exclude_columns: t.Set[str] = set()
pivot = t.cast(t.Optional[exp.Pivot], seq_get(scope.pivots, 0)) pivot = t.cast(t.Optional[exp.Pivot], seq_get(scope.pivots, 0))
if isinstance(pivot, exp.Pivot) and not pivot.alias_column_names: if isinstance(pivot, exp.Pivot) and not pivot.alias_column_names:
if pivot.unpivot: if pivot.unpivot:
pivot_output_columns = [c.output_name for c in _unpivot_columns(pivot)] pivot_output_columns = [c.output_name for c in _unpivot_columns(pivot)]
field = pivot.args.get("field") for field in pivot.fields:
if isinstance(field, exp.In): if isinstance(field, exp.In):
pivot_exclude_columns = { pivot_exclude_columns.update(
c.output_name for e in field.expressions for c in e.find_all(exp.Column) c.output_name for e in field.expressions for c in e.find_all(exp.Column)
} )
else: else:
pivot_exclude_columns = set(c.output_name for c in pivot.find_all(exp.Column)) pivot_exclude_columns = set(c.output_name for c in pivot.find_all(exp.Column))
@ -916,6 +918,32 @@ class Resolver:
if source.expression.is_type(exp.DataType.Type.STRUCT): if source.expression.is_type(exp.DataType.Type.STRUCT):
for k in source.expression.type.expressions: # type: ignore for k in source.expression.type.expressions: # type: ignore
columns.append(k.name) columns.append(k.name)
elif isinstance(source, Scope) and isinstance(source.expression, exp.SetOperation):
set_op = source.expression
# BigQuery specific set operations modifiers, e.g INNER UNION ALL BY NAME
on_column_list = set_op.args.get("on")
if on_column_list:
# The resulting columns are the columns in the ON clause:
# {INNER | LEFT | FULL} UNION ALL BY NAME ON (col1, col2, ...)
columns = [col.name for col in on_column_list]
elif set_op.side or set_op.kind:
side = set_op.side
kind = set_op.kind
left = set_op.left.named_selects
right = set_op.right.named_selects
# We use dict.fromkeys to deduplicate keys and maintain insertion order
if side == "LEFT":
columns = left
elif side == "FULL":
columns = list(dict.fromkeys(left + right))
elif kind == "INNER":
columns = list(dict.fromkeys(left).keys() & dict.fromkeys(right).keys())
else:
columns = set_op.named_selects
else: else:
columns = source.expression.named_selects columns = source.expression.named_selects

6
sqlglot/optimizer/qualify_tables.py
View file

@ -8,6 +8,7 @@ from sqlglot.dialects.dialect import DialectType
from sqlglot.helper import csv_reader, name_sequence from sqlglot.helper import csv_reader, name_sequence
from sqlglot.optimizer.scope import Scope, traverse_scope from sqlglot.optimizer.scope import Scope, traverse_scope
from sqlglot.schema import Schema from sqlglot.schema import Schema
from sqlglot.dialects.dialect import Dialect
if t.TYPE_CHECKING: if t.TYPE_CHECKING:
from sqlglot._typing import E from sqlglot._typing import E
@ -49,6 +50,7 @@ def qualify_tables(
next_alias_name = name_sequence("_q_") next_alias_name = name_sequence("_q_")
db = exp.parse_identifier(db, dialect=dialect) if db else None db = exp.parse_identifier(db, dialect=dialect) if db else None
catalog = exp.parse_identifier(catalog, dialect=dialect) if catalog else None catalog = exp.parse_identifier(catalog, dialect=dialect) if catalog else None
dialect = Dialect.get_or_raise(dialect)
def _qualify(table: exp.Table) -> None: def _qualify(table: exp.Table) -> None:
if isinstance(table.this, exp.Identifier): if isinstance(table.this, exp.Identifier):
@ -127,8 +129,8 @@ def qualify_tables(
if not table_alias.name: if not table_alias.name:
table_alias.set("this", exp.to_identifier(next_alias_name())) table_alias.set("this", exp.to_identifier(next_alias_name()))
if isinstance(udtf, exp.Values) and not table_alias.columns: if isinstance(udtf, exp.Values) and not table_alias.columns:
for i, e in enumerate(udtf.expressions[0].expressions): column_aliases = dialect.generate_values_aliases(udtf)
table_alias.append("columns", exp.to_identifier(f"_col_{i}")) table_alias.set("columns", column_aliases)
else: else:
for node in scope.walk(): for node in scope.walk():
if ( if (

140
sqlglot/parser.py
View file

@ -2,6 +2,7 @@ from __future__ import annotations
import logging import logging
import typing as t import typing as t
import itertools
from collections import defaultdict from collections import defaultdict
from sqlglot import exp from sqlglot import exp
@ -438,12 +439,14 @@ class Parser(metaclass=_Parser):
DB_CREATABLES = { DB_CREATABLES = {
TokenType.DATABASE, TokenType.DATABASE,
TokenType.DICTIONARY, TokenType.DICTIONARY,
TokenType.FILE_FORMAT,
TokenType.MODEL, TokenType.MODEL,
TokenType.NAMESPACE, TokenType.NAMESPACE,
TokenType.SCHEMA, TokenType.SCHEMA,
TokenType.SEQUENCE, TokenType.SEQUENCE,
TokenType.SINK, TokenType.SINK,
TokenType.SOURCE, TokenType.SOURCE,
TokenType.STAGE,
TokenType.STORAGE_INTEGRATION, TokenType.STORAGE_INTEGRATION,
TokenType.STREAMLIT, TokenType.STREAMLIT,
TokenType.TABLE, TokenType.TABLE,
@ -802,7 +805,7 @@ class Parser(metaclass=_Parser):
exp.Sort: lambda self: self._parse_sort(exp.Sort, TokenType.SORT_BY), exp.Sort: lambda self: self._parse_sort(exp.Sort, TokenType.SORT_BY),
exp.Table: lambda self: self._parse_table_parts(), exp.Table: lambda self: self._parse_table_parts(),
exp.TableAlias: lambda self: self._parse_table_alias(), exp.TableAlias: lambda self: self._parse_table_alias(),
exp.Tuple: lambda self: self._parse_value(), exp.Tuple: lambda self: self._parse_value(values=False),
exp.Whens: lambda self: self._parse_when_matched(), exp.Whens: lambda self: self._parse_when_matched(),
exp.Where: lambda self: self._parse_where(), exp.Where: lambda self: self._parse_where(),
exp.Window: lambda self: self._parse_named_window(), exp.Window: lambda self: self._parse_named_window(),
@ -3037,7 +3040,7 @@ class Parser(metaclass=_Parser):
expressions=self._parse_wrapped_csv(self._parse_assignment), expressions=self._parse_wrapped_csv(self._parse_assignment),
) )
def _parse_value(self) -> t.Optional[exp.Tuple]: def _parse_value(self, values: bool = True) -> t.Optional[exp.Tuple]:
def _parse_value_expression() -> t.Optional[exp.Expression]: def _parse_value_expression() -> t.Optional[exp.Expression]:
if self.dialect.SUPPORTS_VALUES_DEFAULT and self._match(TokenType.DEFAULT): if self.dialect.SUPPORTS_VALUES_DEFAULT and self._match(TokenType.DEFAULT):
return exp.var(self._prev.text.upper()) return exp.var(self._prev.text.upper())
@ -3135,7 +3138,7 @@ class Parser(metaclass=_Parser):
if distinct: if distinct:
distinct = self.expression( distinct = self.expression(
exp.Distinct, exp.Distinct,
on=self._parse_value() if self._match(TokenType.ON) else None, on=self._parse_value(values=False) if self._match(TokenType.ON) else None,
) )
if all_ and distinct: if all_ and distinct:
@ -3573,6 +3576,8 @@ class Parser(metaclass=_Parser):
expression=self._parse_function() or self._parse_id_var(any_token=False), expression=self._parse_function() or self._parse_id_var(any_token=False),
) )
ordinality: t.Optional[bool] = None
if view: if view:
table = self._parse_id_var(any_token=False) table = self._parse_id_var(any_token=False)
columns = self._parse_csv(self._parse_id_var) if self._match(TokenType.ALIAS) else [] columns = self._parse_csv(self._parse_id_var) if self._match(TokenType.ALIAS) else []
@ -3583,6 +3588,7 @@ class Parser(metaclass=_Parser):
# We move the alias from the lateral's child node to the lateral itself # We move the alias from the lateral's child node to the lateral itself
table_alias = this.args["alias"].pop() table_alias = this.args["alias"].pop()
else: else:
ordinality = self._match_pair(TokenType.WITH, TokenType.ORDINALITY)
table_alias = self._parse_table_alias() table_alias = self._parse_table_alias()
return self.expression( return self.expression(
@ -3592,6 +3598,7 @@ class Parser(metaclass=_Parser):
outer=outer, outer=outer,
alias=table_alias, alias=table_alias,
cross_apply=cross_apply, cross_apply=cross_apply,
ordinality=ordinality,
) )
def _parse_join_parts( def _parse_join_parts(
@ -4180,7 +4187,7 @@ class Parser(metaclass=_Parser):
into=into, into=into,
) )
def _parse_pivot_in(self) -> exp.In | exp.PivotAny: def _parse_pivot_in(self) -> exp.In:
def _parse_aliased_expression() -> t.Optional[exp.Expression]: def _parse_aliased_expression() -> t.Optional[exp.Expression]:
this = self._parse_select_or_expression() this = self._parse_select_or_expression()
@ -4240,20 +4247,29 @@ class Parser(metaclass=_Parser):
if not self._match(TokenType.FOR): if not self._match(TokenType.FOR):
self.raise_error("Expecting FOR") self.raise_error("Expecting FOR")
field = self._parse_pivot_in() fields = []
while True:
field = self._try_parse(self._parse_pivot_in)
if not field:
break
fields.append(field)
default_on_null = self._match_text_seq("DEFAULT", "ON", "NULL") and self._parse_wrapped( default_on_null = self._match_text_seq("DEFAULT", "ON", "NULL") and self._parse_wrapped(
self._parse_bitwise self._parse_bitwise
) )
group = self._parse_group()
self._match_r_paren() self._match_r_paren()
pivot = self.expression( pivot = self.expression(
exp.Pivot, exp.Pivot,
expressions=expressions, expressions=expressions,
field=field, fields=fields,
unpivot=unpivot, unpivot=unpivot,
include_nulls=include_nulls, include_nulls=include_nulls,
default_on_null=default_on_null, default_on_null=default_on_null,
group=group,
) )
if not self._match_set((TokenType.PIVOT, TokenType.UNPIVOT), advance=False): if not self._match_set((TokenType.PIVOT, TokenType.UNPIVOT), advance=False):
@ -4263,26 +4279,43 @@ class Parser(metaclass=_Parser):
names = self._pivot_column_names(t.cast(t.List[exp.Expression], expressions)) names = self._pivot_column_names(t.cast(t.List[exp.Expression], expressions))
columns: t.List[exp.Expression] = [] columns: t.List[exp.Expression] = []
pivot_field_expressions = pivot.args["field"].expressions all_fields = []
for pivot_field in pivot.fields:
pivot_field_expressions = pivot_field.expressions
# The `PivotAny` expression corresponds to `ANY ORDER BY <column>`; we can't infer in this case. # The `PivotAny` expression corresponds to `ANY ORDER BY <column>`; we can't infer in this case.
if not isinstance(seq_get(pivot_field_expressions, 0), exp.PivotAny): if isinstance(seq_get(pivot_field_expressions, 0), exp.PivotAny):
for fld in pivot_field_expressions: continue
field_name = fld.sql() if self.IDENTIFY_PIVOT_STRINGS else fld.alias_or_name
for name in names:
if self.PREFIXED_PIVOT_COLUMNS:
name = f"{name}_{field_name}" if name else field_name
else:
name = f"{field_name}_{name}" if name else field_name
columns.append(exp.to_identifier(name)) all_fields.append(
[
fld.sql() if self.IDENTIFY_PIVOT_STRINGS else fld.alias_or_name
for fld in pivot_field_expressions
]
)
if all_fields:
if names:
all_fields.append(names)
# Generate all possible combinations of the pivot columns
# e.g PIVOT(sum(...) as total FOR year IN (2000, 2010) FOR country IN ('NL', 'US'))
# generates the product between [[2000, 2010], ['NL', 'US'], ['total']]
for fld_parts_tuple in itertools.product(*all_fields):
fld_parts = list(fld_parts_tuple)
if names and self.PREFIXED_PIVOT_COLUMNS:
# Move the "name" to the front of the list
fld_parts.insert(0, fld_parts.pop(-1))
columns.append(exp.to_identifier("_".join(fld_parts)))
pivot.set("columns", columns) pivot.set("columns", columns)
return pivot return pivot
def _pivot_column_names(self, aggregations: t.List[exp.Expression]) -> t.List[str]: def _pivot_column_names(self, aggregations: t.List[exp.Expression]) -> t.List[str]:
return [agg.alias for agg in aggregations] return [agg.alias for agg in aggregations if agg.alias]
def _parse_prewhere(self, skip_where_token: bool = False) -> t.Optional[exp.PreWhere]: def _parse_prewhere(self, skip_where_token: bool = False) -> t.Optional[exp.PreWhere]:
if not skip_where_token and not self._match(TokenType.PREWHERE): if not skip_where_token and not self._match(TokenType.PREWHERE):
@ -4591,8 +4624,17 @@ class Parser(metaclass=_Parser):
return locks return locks
def _parse_set_operations(self, this: t.Optional[exp.Expression]) -> t.Optional[exp.Expression]: def parse_set_operation(self, this: t.Optional[exp.Expression]) -> t.Optional[exp.Expression]:
while this and self._match_set(self.SET_OPERATIONS): start = self._index
_, side_token, kind_token = self._parse_join_parts()
side = side_token.text if side_token else None
kind = kind_token.text if kind_token else None
if not self._match_set(self.SET_OPERATIONS):
self._retreat(start)
return None
token_type = self._prev.token_type token_type = self._prev.token_type
if token_type == TokenType.UNION: if token_type == TokenType.UNION:
@ -4613,18 +4655,39 @@ class Parser(metaclass=_Parser):
if distinct is None: if distinct is None:
self.raise_error(f"Expected DISTINCT or ALL for {operation.__name__}") self.raise_error(f"Expected DISTINCT or ALL for {operation.__name__}")
by_name = self._match_text_seq("BY", "NAME") by_name = self._match_text_seq("BY", "NAME") or self._match_text_seq(
"STRICT", "CORRESPONDING"
)
if self._match_text_seq("CORRESPONDING"):
by_name = True
if not side and not kind:
kind = "INNER"
on_column_list = None
if by_name and self._match_texts(("ON", "BY")):
on_column_list = self._parse_wrapped_csv(self._parse_column)
expression = self._parse_select(nested=True, parse_set_operation=False) expression = self._parse_select(nested=True, parse_set_operation=False)
this = self.expression( return self.expression(
operation, operation,
comments=comments, comments=comments,
this=this, this=this,
distinct=distinct, distinct=distinct,
by_name=by_name, by_name=by_name,
expression=expression, expression=expression,
side=side,
kind=kind,
on=on_column_list,
) )
def _parse_set_operations(self, this: t.Optional[exp.Expression]) -> t.Optional[exp.Expression]:
while True:
setop = self.parse_set_operation(this)
if not setop:
break
this = setop
if isinstance(this, exp.SetOperation) and self.MODIFIERS_ATTACHED_TO_SET_OP: if isinstance(this, exp.SetOperation) and self.MODIFIERS_ATTACHED_TO_SET_OP:
expression = this.expression expression = this.expression
@ -6090,7 +6153,9 @@ class Parser(metaclass=_Parser):
dialect=self.dialect, dialect=self.dialect,
) )
expressions = apply_index_offset(this, expressions, -self.dialect.INDEX_OFFSET) expressions = apply_index_offset(
this, expressions, -self.dialect.INDEX_OFFSET, dialect=self.dialect
)
this = self.expression(exp.Bracket, this=this, expressions=expressions) this = self.expression(exp.Bracket, this=this, expressions=expressions)
self._add_comments(this) self._add_comments(this)
@ -6129,7 +6194,9 @@ class Parser(metaclass=_Parser):
def _parse_if(self) -> t.Optional[exp.Expression]: def _parse_if(self) -> t.Optional[exp.Expression]:
if self._match(TokenType.L_PAREN): if self._match(TokenType.L_PAREN):
args = self._parse_csv(self._parse_assignment) args = self._parse_csv(
lambda: self._parse_alias(self._parse_assignment(), explicit=True)
)
this = self.validate_expression(exp.If.from_arg_list(args), args) this = self.validate_expression(exp.If.from_arg_list(args), args)
self._match_r_paren() self._match_r_paren()
else: else:
@ -7490,7 +7557,9 @@ class Parser(metaclass=_Parser):
else: else:
then = self.expression( then = self.expression(
exp.Insert, exp.Insert,
this=exp.var("ROW") if self._match_text_seq("ROW") else self._parse_value(), this=exp.var("ROW")
if self._match_text_seq("ROW")
else self._parse_value(values=False),
expression=self._match_text_seq("VALUES") and self._parse_value(), expression=self._match_text_seq("VALUES") and self._parse_value(),
) )
elif self._match(TokenType.UPDATE): elif self._match(TokenType.UPDATE):
@ -7899,17 +7968,19 @@ class Parser(metaclass=_Parser):
self._match(TokenType.L_PAREN) self._match(TokenType.L_PAREN)
opts: t.List[t.Optional[exp.Expression]] = [] opts: t.List[t.Optional[exp.Expression]] = []
option: exp.Expression | None
while self._curr and not self._match(TokenType.R_PAREN): while self._curr and not self._match(TokenType.R_PAREN):
if self._match_text_seq("FORMAT_NAME", "="): if self._match_text_seq("FORMAT_NAME", "="):
# The FORMAT_NAME can be set to an identifier for Snowflake and T-SQL # The FORMAT_NAME can be set to an identifier for Snowflake and T-SQL
prop = self.expression( option = self._parse_format_name()
exp.Property, this=exp.var("FORMAT_NAME"), value=self._parse_table_parts()
)
opts.append(prop)
else: else:
opts.append(self._parse_property()) option = self._parse_property()
self._match(TokenType.COMMA) if option is None:
self.raise_error("Unable to parse option")
break
opts.append(option)
return opts return opts
@ -8100,3 +8171,12 @@ class Parser(metaclass=_Parser):
"for": self._match_text_seq("FOR") and self._parse_bitwise(), "for": self._match_text_seq("FOR") and self._parse_bitwise(),
}, },
) )
def _parse_format_name(self) -> exp.Property:
# Note: Although not specified in the docs, Snowflake does accept a string/identifier
# for FILE_FORMAT = <format_name>
return self.expression(
exp.Property,
this=exp.var("FORMAT_NAME"),
value=self._parse_string() or self._parse_table_parts(),
)

2
sqlglot/tokens.py
View file

@ -97,6 +97,7 @@ class TokenType(AutoName):
SCHEMA = auto() SCHEMA = auto()
TABLE = auto() TABLE = auto()
WAREHOUSE = auto() WAREHOUSE = auto()
STAGE = auto()
STREAMLIT = auto() STREAMLIT = auto()
VAR = auto() VAR = auto()
BIT_STRING = auto() BIT_STRING = auto()
@ -277,6 +278,7 @@ class TokenType(AutoName):
EXISTS = auto() EXISTS = auto()
FALSE = auto() FALSE = auto()
FETCH = auto() FETCH = auto()
FILE_FORMAT = auto()
FILTER = auto() FILTER = auto()
FINAL = auto() FINAL = auto()
FIRST = auto() FIRST = auto()

6
tests/dialects/test_athena.py
View file

@ -154,6 +154,12 @@ class TestAthena(Validator):
write_sql='CREATE TABLE "foo" AS WITH "foo" AS (SELECT "a", "b" FROM "bar") SELECT * FROM "foo"', write_sql='CREATE TABLE "foo" AS WITH "foo" AS (SELECT "a", "b" FROM "bar") SELECT * FROM "foo"',
) )
# CTAS with Union should still hit the Trino engine and not Hive
self.validate_identity(
'CREATE TABLE `foo` AS WITH `foo` AS (SELECT "a", `b` FROM "bar") SELECT * FROM "foo" UNION SELECT * FROM "foo"',
write_sql='CREATE TABLE "foo" AS WITH "foo" AS (SELECT "a", "b" FROM "bar") SELECT * FROM "foo" UNION SELECT * FROM "foo"',
)
self.validate_identity("DESCRIBE foo.bar", write_sql="DESCRIBE `foo`.`bar`", identify=True) self.validate_identity("DESCRIBE foo.bar", write_sql="DESCRIBE `foo`.`bar`", identify=True)
def test_dml_quoting(self): def test_dml_quoting(self):

48
tests/dialects/test_bigquery.py
View file

@ -1404,11 +1404,13 @@ LANGUAGE js AS
) )
self.validate_all( self.validate_all(
"TO_JSON_STRING(x)", "TO_JSON_STRING(x)",
read={"bigquery": "TO_JSON_STRING(x)"}, read={
"bigquery": "TO_JSON_STRING(x)",
},
write={ write={
"bigquery": "TO_JSON_STRING(x)", "bigquery": "TO_JSON_STRING(x)",
"duckdb": "CAST(TO_JSON(x) AS TEXT)", "duckdb": "CAST(TO_JSON(x) AS TEXT)",
"presto": "JSON_FORMAT(x)", "presto": "JSON_FORMAT(CAST(x AS JSON))",
"spark": "TO_JSON(x)", "spark": "TO_JSON(x)",
}, },
) )
@ -1486,7 +1488,7 @@ WHERE
"snowflake": "SELECT LENGTH(foo)", "snowflake": "SELECT LENGTH(foo)",
}, },
write={ write={
"duckdb": "SELECT CASE TYPEOF(foo) WHEN 'VARCHAR' THEN LENGTH(CAST(foo AS TEXT)) WHEN 'BLOB' THEN OCTET_LENGTH(CAST(foo AS BLOB)) END", "duckdb": "SELECT CASE TYPEOF(foo) WHEN 'BLOB' THEN OCTET_LENGTH(CAST(foo AS BLOB)) ELSE LENGTH(CAST(foo AS TEXT)) END",
"snowflake": "SELECT LENGTH(foo)", "snowflake": "SELECT LENGTH(foo)",
"": "SELECT LENGTH(foo)", "": "SELECT LENGTH(foo)",
}, },
@ -2382,3 +2384,43 @@ OPTIONS (
for select in annotated.selects: for select in annotated.selects:
self.assertEqual(select.type.sql("bigquery"), "TIMESTAMP") self.assertEqual(select.type.sql("bigquery"), "TIMESTAMP")
def test_set_operations(self):
self.validate_identity("SELECT 1 AS foo INNER UNION ALL SELECT 3 AS foo, 4 AS bar")
for side in ("", " LEFT", " FULL"):
for kind in ("", " OUTER"):
for name in (
"",
" BY NAME",
" BY NAME ON (foo, bar)",
):
with self.subTest(f"Testing {side} {kind} {name} in test_set_operations"):
self.validate_identity(
f"SELECT 1 AS foo{side}{kind} UNION ALL{name} SELECT 3 AS foo, 4 AS bar",
)
self.validate_identity(
"SELECT 1 AS x UNION ALL CORRESPONDING SELECT 2 AS x",
"SELECT 1 AS x INNER UNION ALL BY NAME SELECT 2 AS x",
)
self.validate_identity(
"SELECT 1 AS x UNION ALL CORRESPONDING BY (foo, bar) SELECT 2 AS x",
"SELECT 1 AS x INNER UNION ALL BY NAME ON (foo, bar) SELECT 2 AS x",
)
self.validate_identity(
"SELECT 1 AS x LEFT UNION ALL CORRESPONDING SELECT 2 AS x",
"SELECT 1 AS x LEFT UNION ALL BY NAME SELECT 2 AS x",
)
self.validate_identity(
"SELECT 1 AS x UNION ALL STRICT CORRESPONDING SELECT 2 AS x",
"SELECT 1 AS x UNION ALL BY NAME SELECT 2 AS x",
)
self.validate_identity(
"SELECT 1 AS x UNION ALL STRICT CORRESPONDING BY (foo, bar) SELECT 2 AS x",
"SELECT 1 AS x UNION ALL BY NAME ON (foo, bar) SELECT 2 AS x",
)

50
tests/dialects/test_clickhouse.py
View file

@ -33,6 +33,7 @@ class TestClickhouse(Validator):
self.assertEqual(expr.sql(dialect="clickhouse"), "COUNT(x)") self.assertEqual(expr.sql(dialect="clickhouse"), "COUNT(x)")
self.assertIsNone(expr._meta) self.assertIsNone(expr._meta)
self.validate_identity('SELECT DISTINCT ON ("id") * FROM t')
self.validate_identity("SELECT 1 OR (1 = 2)") self.validate_identity("SELECT 1 OR (1 = 2)")
self.validate_identity("SELECT 1 AND (1 = 2)") self.validate_identity("SELECT 1 AND (1 = 2)")
self.validate_identity("SELECT json.a.:Int64") self.validate_identity("SELECT json.a.:Int64")
@ -105,8 +106,10 @@ class TestClickhouse(Validator):
self.validate_identity("SELECT * FROM table LIMIT 1 BY a, b") self.validate_identity("SELECT * FROM table LIMIT 1 BY a, b")
self.validate_identity("SELECT * FROM table LIMIT 2 OFFSET 1 BY a, b") self.validate_identity("SELECT * FROM table LIMIT 2 OFFSET 1 BY a, b")
self.validate_identity("TRUNCATE TABLE t1 ON CLUSTER test_cluster") self.validate_identity("TRUNCATE TABLE t1 ON CLUSTER test_cluster")
self.validate_identity("TRUNCATE TABLE t1 ON CLUSTER '{cluster}'")
self.validate_identity("TRUNCATE DATABASE db") self.validate_identity("TRUNCATE DATABASE db")
self.validate_identity("TRUNCATE DATABASE db ON CLUSTER test_cluster") self.validate_identity("TRUNCATE DATABASE db ON CLUSTER test_cluster")
self.validate_identity("TRUNCATE DATABASE db ON CLUSTER '{cluster}'")
self.validate_identity( self.validate_identity(
"SELECT DATE_BIN(toDateTime('2023-01-01 14:45:00'), INTERVAL '1' MINUTE, toDateTime('2023-01-01 14:35:30'), 'UTC')", "SELECT DATE_BIN(toDateTime('2023-01-01 14:45:00'), INTERVAL '1' MINUTE, toDateTime('2023-01-01 14:35:30'), 'UTC')",
) )
@ -155,12 +158,21 @@ class TestClickhouse(Validator):
self.validate_identity( self.validate_identity(
"CREATE TABLE test ON CLUSTER default (id UInt8) ENGINE=AggregatingMergeTree() ORDER BY tuple()" "CREATE TABLE test ON CLUSTER default (id UInt8) ENGINE=AggregatingMergeTree() ORDER BY tuple()"
) )
self.validate_identity(
"CREATE TABLE test ON CLUSTER '{cluster}' (id UInt8) ENGINE=AggregatingMergeTree() ORDER BY tuple()"
)
self.validate_identity( self.validate_identity(
"CREATE MATERIALIZED VIEW test_view ON CLUSTER cl1 (id UInt8) ENGINE=AggregatingMergeTree() ORDER BY tuple() AS SELECT * FROM test_data" "CREATE MATERIALIZED VIEW test_view ON CLUSTER cl1 (id UInt8) ENGINE=AggregatingMergeTree() ORDER BY tuple() AS SELECT * FROM test_data"
) )
self.validate_identity(
"CREATE MATERIALIZED VIEW test_view ON CLUSTER '{cluster}' (id UInt8) ENGINE=AggregatingMergeTree() ORDER BY tuple() AS SELECT * FROM test_data"
)
self.validate_identity( self.validate_identity(
"CREATE MATERIALIZED VIEW test_view ON CLUSTER cl1 TO table1 AS SELECT * FROM test_data" "CREATE MATERIALIZED VIEW test_view ON CLUSTER cl1 TO table1 AS SELECT * FROM test_data"
) )
self.validate_identity(
"CREATE MATERIALIZED VIEW test_view ON CLUSTER '{cluster}' TO table1 AS SELECT * FROM test_data"
)
self.validate_identity( self.validate_identity(
"CREATE MATERIALIZED VIEW test_view TO db.table1 (id UInt8) AS SELECT * FROM test_data" "CREATE MATERIALIZED VIEW test_view TO db.table1 (id UInt8) AS SELECT * FROM test_data"
) )
@ -184,7 +196,7 @@ class TestClickhouse(Validator):
) )
self.validate_identity( self.validate_identity(
"INSERT INTO tab VALUES ({'key1': 1, 'key2': 10}), ({'key1': 2, 'key2': 20}), ({'key1': 3, 'key2': 30})", "INSERT INTO tab VALUES ({'key1': 1, 'key2': 10}), ({'key1': 2, 'key2': 20}), ({'key1': 3, 'key2': 30})",
"INSERT INTO tab VALUES (map('key1', 1, 'key2', 10)), (map('key1', 2, 'key2', 20)), (map('key1', 3, 'key2', 30))", "INSERT INTO tab VALUES ((map('key1', 1, 'key2', 10))), ((map('key1', 2, 'key2', 20))), ((map('key1', 3, 'key2', 30)))",
) )
self.validate_identity( self.validate_identity(
"SELECT (toUInt8('1') + toUInt8('2')) IS NOT NULL", "SELECT (toUInt8('1') + toUInt8('2')) IS NOT NULL",
@ -507,11 +519,12 @@ class TestClickhouse(Validator):
"INSERT INTO FUNCTION s3('url', 'CSV', 'name String, value UInt32', 'gzip') SELECT name, value FROM existing_table" "INSERT INTO FUNCTION s3('url', 'CSV', 'name String, value UInt32', 'gzip') SELECT name, value FROM existing_table"
) )
self.validate_identity( self.validate_identity(
"INSERT INTO FUNCTION remote('localhost', default.simple_table) VALUES (100, 'inserted via remote()')" "INSERT INTO FUNCTION remote('localhost', default.simple_table) VALUES (100, 'inserted via remote()')",
"INSERT INTO FUNCTION remote('localhost', default.simple_table) VALUES ((100), ('inserted via remote()'))",
) )
self.validate_identity( self.validate_identity(
"""INSERT INTO TABLE FUNCTION hdfs('hdfs://hdfs1:9000/test', 'TSV', 'name String, column2 UInt32, column3 UInt32') VALUES ('test', 1, 2)""", """INSERT INTO TABLE FUNCTION hdfs('hdfs://hdfs1:9000/test', 'TSV', 'name String, column2 UInt32, column3 UInt32') VALUES ('test', 1, 2)""",
"""INSERT INTO FUNCTION hdfs('hdfs://hdfs1:9000/test', 'TSV', 'name String, column2 UInt32, column3 UInt32') VALUES ('test', 1, 2)""", """INSERT INTO FUNCTION hdfs('hdfs://hdfs1:9000/test', 'TSV', 'name String, column2 UInt32, column3 UInt32') VALUES (('test'), (1), (2))""",
) )
self.validate_identity("SELECT 1 FORMAT TabSeparated") self.validate_identity("SELECT 1 FORMAT TabSeparated")
@ -546,22 +559,23 @@ class TestClickhouse(Validator):
) )
self.validate_identity("ALTER TABLE visits REPLACE PARTITION ID '201901' FROM visits_tmp") self.validate_identity("ALTER TABLE visits REPLACE PARTITION ID '201901' FROM visits_tmp")
self.validate_identity("ALTER TABLE visits ON CLUSTER test_cluster DROP COLUMN col1") self.validate_identity("ALTER TABLE visits ON CLUSTER test_cluster DROP COLUMN col1")
self.validate_identity("ALTER TABLE visits ON CLUSTER '{cluster}' DROP COLUMN col1")
self.validate_identity("DELETE FROM tbl ON CLUSTER test_cluster WHERE date = '2019-01-01'") self.validate_identity("DELETE FROM tbl ON CLUSTER test_cluster WHERE date = '2019-01-01'")
self.validate_identity("DELETE FROM tbl ON CLUSTER '{cluster}' WHERE date = '2019-01-01'")
self.assertIsInstance( self.assertIsInstance(
parse_one("Tuple(select Int64)", into=exp.DataType, read="clickhouse"), exp.DataType parse_one("Tuple(select Int64)", into=exp.DataType, read="clickhouse"), exp.DataType
) )
self.validate_identity("INSERT INTO t (col1, col2) VALUES ('abcd', 1234)") self.validate_identity(
"INSERT INTO t (col1, col2) VALUES ('abcd', 1234)",
"INSERT INTO t (col1, col2) VALUES (('abcd'), (1234))",
)
self.validate_all( self.validate_all(
"INSERT INTO t (col1, col2) VALUES ('abcd', 1234)", "INSERT INTO t (col1, col2) VALUES ('abcd', 1234)",
read={
# looks like values table function, but should be parsed as VALUES block
"clickhouse": "INSERT INTO t (col1, col2) values('abcd', 1234)"
},
write={ write={
"clickhouse": "INSERT INTO t (col1, col2) VALUES ('abcd', 1234)", "clickhouse": "INSERT INTO t (col1, col2) VALUES (('abcd'), (1234))",
"postgres": "INSERT INTO t (col1, col2) VALUES ('abcd', 1234)", "postgres": "INSERT INTO t (col1, col2) VALUES (('abcd'), (1234))",
}, },
) )
self.validate_identity("SELECT TRIM(TRAILING ')' FROM '( Hello, world! )')") self.validate_identity("SELECT TRIM(TRAILING ')' FROM '( Hello, world! )')")
@ -591,6 +605,9 @@ class TestClickhouse(Validator):
self.validate_identity("SELECT arrayConcat([1, 2], [3, 4])") self.validate_identity("SELECT arrayConcat([1, 2], [3, 4])")
def test_clickhouse_values(self): def test_clickhouse_values(self):
ast = self.parse_one("SELECT * FROM VALUES (1, 2, 3)")
self.assertEqual(len(list(ast.find_all(exp.Tuple))), 4)
values = exp.select("*").from_( values = exp.select("*").from_(
exp.values([exp.tuple_(1, 2, 3)], alias="subq", columns=["a", "b", "c"]) exp.values([exp.tuple_(1, 2, 3)], alias="subq", columns=["a", "b", "c"])
) )
@ -599,10 +616,18 @@ class TestClickhouse(Validator):
"SELECT * FROM (SELECT 1 AS a, 2 AS b, 3 AS c) AS subq", "SELECT * FROM (SELECT 1 AS a, 2 AS b, 3 AS c) AS subq",
) )
self.validate_identity("INSERT INTO t (col1, col2) VALUES ('abcd', 1234)") self.validate_identity("SELECT * FROM VALUES ((1, 1), (2, 1), (3, 1), (4, 1))")
self.validate_identity(
"SELECT type, id FROM VALUES ('id Int, type Int', (1, 1), (2, 1), (3, 1), (4, 1))"
)
self.validate_identity(
"INSERT INTO t (col1, col2) VALUES ('abcd', 1234)",
"INSERT INTO t (col1, col2) VALUES (('abcd'), (1234))",
)
self.validate_identity( self.validate_identity(
"INSERT INTO t (col1, col2) FORMAT Values('abcd', 1234)", "INSERT INTO t (col1, col2) FORMAT Values('abcd', 1234)",
"INSERT INTO t (col1, col2) VALUES ('abcd', 1234)", "INSERT INTO t (col1, col2) VALUES (('abcd'), (1234))",
) )
self.validate_all( self.validate_all(
@ -1171,6 +1196,7 @@ LIFETIME(MIN 0 MAX 0)""",
for creatable in ("DATABASE", "TABLE", "VIEW", "DICTIONARY", "FUNCTION"): for creatable in ("DATABASE", "TABLE", "VIEW", "DICTIONARY", "FUNCTION"):
with self.subTest(f"Test DROP {creatable} ON CLUSTER"): with self.subTest(f"Test DROP {creatable} ON CLUSTER"):
self.validate_identity(f"DROP {creatable} test ON CLUSTER test_cluster") self.validate_identity(f"DROP {creatable} test ON CLUSTER test_cluster")
self.validate_identity(f"DROP {creatable} test ON CLUSTER '{{cluster}}'")
def test_datetime_funcs(self): def test_datetime_funcs(self):
# Each datetime func has an alias that is roundtripped to the original name e.g. (DATE_SUB, DATESUB) -> DATE_SUB # Each datetime func has an alias that is roundtripped to the original name e.g. (DATE_SUB, DATESUB) -> DATE_SUB

18
tests/dialects/test_databricks.py
View file

@ -68,6 +68,24 @@ class TestDatabricks(Validator):
"FROM_UTC_TIMESTAMP(CAST(x AS TIMESTAMP), tz)", "FROM_UTC_TIMESTAMP(CAST(x AS TIMESTAMP), tz)",
) )
self.validate_all(
"SELECT c1:item[1].price",
read={
"spark": "SELECT GET_JSON_OBJECT(c1, '$.item[1].price')",
},
write={
"databricks": "SELECT c1:item[1].price",
"spark": "SELECT GET_JSON_OBJECT(c1, '$.item[1].price')",
},
)
self.validate_all(
"SELECT GET_JSON_OBJECT(c1, '$.item[1].price')",
write={
"databricks": "SELECT c1:item[1].price",
"spark": "SELECT GET_JSON_OBJECT(c1, '$.item[1].price')",
},
)
self.validate_all( self.validate_all(
"CREATE TABLE foo (x INT GENERATED ALWAYS AS (YEAR(y)))", "CREATE TABLE foo (x INT GENERATED ALWAYS AS (YEAR(y)))",
write={ write={

7
tests/dialects/test_duckdb.py
View file

@ -400,6 +400,9 @@ class TestDuckDB(Validator):
self.validate_identity( self.validate_identity(
"SELECT * FROM (PIVOT Cities ON Year USING SUM(Population) GROUP BY Country) AS pivot_alias" "SELECT * FROM (PIVOT Cities ON Year USING SUM(Population) GROUP BY Country) AS pivot_alias"
) )
self.validate_identity(
"SELECT * FROM cities PIVOT(SUM(population) FOR year IN (2000, 2010, 2020) GROUP BY country)"
)
self.validate_identity( self.validate_identity(
# QUALIFY comes after WINDOW # QUALIFY comes after WINDOW
"SELECT schema_name, function_name, ROW_NUMBER() OVER my_window AS function_rank FROM DUCKDB_FUNCTIONS() WINDOW my_window AS (PARTITION BY schema_name ORDER BY function_name) QUALIFY ROW_NUMBER() OVER my_window < 3" "SELECT schema_name, function_name, ROW_NUMBER() OVER my_window AS function_rank FROM DUCKDB_FUNCTIONS() WINDOW my_window AS (PARTITION BY schema_name ORDER BY function_name) QUALIFY ROW_NUMBER() OVER my_window < 3"
@ -1595,3 +1598,7 @@ class TestDuckDB(Validator):
"SELECT (@-1) + 1", "SELECT (@-1) + 1",
"SELECT (ABS(-1)) + 1", "SELECT (ABS(-1)) + 1",
) )
def test_show_tables(self):
self.validate_identity("SHOW TABLES").assert_is(exp.Show)
self.validate_identity("SHOW ALL TABLES").assert_is(exp.Show)

3
tests/dialects/test_postgres.py
View file

@ -76,6 +76,9 @@ class TestPostgres(Validator):
self.validate_identity("SELECT CURRENT_SCHEMA") self.validate_identity("SELECT CURRENT_SCHEMA")
self.validate_identity("SELECT CURRENT_USER") self.validate_identity("SELECT CURRENT_USER")
self.validate_identity("SELECT * FROM ONLY t1") self.validate_identity("SELECT * FROM ONLY t1")
self.validate_identity(
"SELECT * FROM test_data, LATERAL JSONB_ARRAY_ELEMENTS(data) WITH ORDINALITY AS elem(value, ordinality)"
)
self.validate_identity( self.validate_identity(
"SELECT id, name FROM xml_data AS t, XMLTABLE('/root/user' PASSING t.xml COLUMNS id INT PATH '@id', name TEXT PATH 'name/text()') AS x" "SELECT id, name FROM xml_data AS t, XMLTABLE('/root/user' PASSING t.xml COLUMNS id INT PATH '@id', name TEXT PATH 'name/text()') AS x"
) )

3
tests/dialects/test_presto.py
View file

@ -1021,9 +1021,6 @@ class TestPresto(Validator):
) )
self.validate_all( self.validate_all(
"JSON_FORMAT(x)", "JSON_FORMAT(x)",
read={
"spark": "TO_JSON(x)",
},
write={ write={
"bigquery": "TO_JSON_STRING(x)", "bigquery": "TO_JSON_STRING(x)",
"duckdb": "CAST(TO_JSON(x) AS TEXT)", "duckdb": "CAST(TO_JSON(x) AS TEXT)",

48
tests/dialects/test_snowflake.py
View file

@ -1,6 +1,6 @@
from unittest import mock from unittest import mock
from sqlglot import UnsupportedError, exp, parse_one from sqlglot import UnsupportedError, exp, parse_one, ParseError
from sqlglot.optimizer.normalize_identifiers import normalize_identifiers from sqlglot.optimizer.normalize_identifiers import normalize_identifiers
from sqlglot.optimizer.qualify_columns import quote_identifiers from sqlglot.optimizer.qualify_columns import quote_identifiers
from tests.dialects.test_dialect import Validator from tests.dialects.test_dialect import Validator
@ -1481,9 +1481,30 @@ class TestSnowflake(Validator):
self.validate_identity("CREATE TAG cost_center ALLOWED_VALUES 'a', 'b'") self.validate_identity("CREATE TAG cost_center ALLOWED_VALUES 'a', 'b'")
self.validate_identity("CREATE WAREHOUSE x").this.assert_is(exp.Identifier) self.validate_identity("CREATE WAREHOUSE x").this.assert_is(exp.Identifier)
self.validate_identity("CREATE STREAMLIT x").this.assert_is(exp.Identifier) self.validate_identity("CREATE STREAMLIT x").this.assert_is(exp.Identifier)
self.validate_identity(
"CREATE TEMPORARY STAGE stage1 FILE_FORMAT=(TYPE=PARQUET)"
).this.assert_is(exp.Table)
self.validate_identity(
"CREATE STAGE stage1 FILE_FORMAT='format1'",
"CREATE STAGE stage1 FILE_FORMAT=(FORMAT_NAME='format1')",
)
self.validate_identity("CREATE STAGE stage1 FILE_FORMAT=(FORMAT_NAME=stage1.format1)")
self.validate_identity("CREATE STAGE stage1 FILE_FORMAT=(FORMAT_NAME='stage1.format1')")
self.validate_identity(
"CREATE STAGE stage1 FILE_FORMAT=schema1.format1",
"CREATE STAGE stage1 FILE_FORMAT=(FORMAT_NAME=schema1.format1)",
)
with self.assertRaises(ParseError):
self.parse_one("CREATE STAGE stage1 FILE_FORMAT=123", dialect="snowflake")
self.validate_identity(
"CREATE STAGE s1 URL='s3://bucket-123' FILE_FORMAT=(TYPE='JSON') CREDENTIALS=(aws_key_id='test' aws_secret_key='test')"
)
self.validate_identity( self.validate_identity(
"CREATE OR REPLACE TAG IF NOT EXISTS cost_center COMMENT='cost_center tag'" "CREATE OR REPLACE TAG IF NOT EXISTS cost_center COMMENT='cost_center tag'"
).this.assert_is(exp.Identifier) ).this.assert_is(exp.Identifier)
self.validate_identity(
"CREATE TEMPORARY FILE FORMAT fileformat1 TYPE=PARQUET COMPRESSION=auto"
).this.assert_is(exp.Table)
self.validate_identity( self.validate_identity(
"CREATE DYNAMIC TABLE product (pre_tax_profit, taxes, after_tax_profit) TARGET_LAG='20 minutes' WAREHOUSE=mywh AS SELECT revenue - cost, (revenue - cost) * tax_rate, (revenue - cost) * (1.0 - tax_rate) FROM staging_table" "CREATE DYNAMIC TABLE product (pre_tax_profit, taxes, after_tax_profit) TARGET_LAG='20 minutes' WAREHOUSE=mywh AS SELECT revenue - cost, (revenue - cost) * tax_rate, (revenue - cost) * (1.0 - tax_rate) FROM staging_table"
) )
@ -1499,9 +1520,6 @@ class TestSnowflake(Validator):
self.validate_identity( self.validate_identity(
"CREATE TABLE orders_clone_restore CLONE orders BEFORE (STATEMENT => '8e5d0ca9-005e-44e6-b858-a8f5b37c5726')" "CREATE TABLE orders_clone_restore CLONE orders BEFORE (STATEMENT => '8e5d0ca9-005e-44e6-b858-a8f5b37c5726')"
) )
self.validate_identity(
"CREATE TABLE a (x DATE, y BIGINT) PARTITION BY (x) integration='q' auto_refresh=TRUE file_format=(type = parquet)"
)
self.validate_identity( self.validate_identity(
"CREATE SCHEMA mytestschema_clone_restore CLONE testschema BEFORE (TIMESTAMP => TO_TIMESTAMP(40 * 365 * 86400))" "CREATE SCHEMA mytestschema_clone_restore CLONE testschema BEFORE (TIMESTAMP => TO_TIMESTAMP(40 * 365 * 86400))"
) )
@ -1544,8 +1562,8 @@ class TestSnowflake(Validator):
partition by (col1,col2,col3) partition by (col1,col2,col3)
location=@s2/logs/ location=@s2/logs/
partition_type = user_specified partition_type = user_specified
file_format = (type = parquet)""", file_format = (type = parquet compression = gzip binary_as_text = false)""",
"CREATE EXTERNAL TABLE et2 (col1 DATE AS (CAST(GET_PATH(PARSE_JSON(metadata$external_table_partition), 'COL1') AS DATE)), col2 VARCHAR AS (CAST(GET_PATH(PARSE_JSON(metadata$external_table_partition), 'COL2') AS VARCHAR)), col3 DECIMAL(38, 0) AS (CAST(GET_PATH(PARSE_JSON(metadata$external_table_partition), 'COL3') AS DECIMAL(38, 0)))) LOCATION @s2/logs/ PARTITION BY (col1, col2, col3) partition_type=user_specified file_format=(type = parquet)", "CREATE EXTERNAL TABLE et2 (col1 DATE AS (CAST(GET_PATH(PARSE_JSON(metadata$external_table_partition), 'COL1') AS DATE)), col2 VARCHAR AS (CAST(GET_PATH(PARSE_JSON(metadata$external_table_partition), 'COL2') AS VARCHAR)), col3 DECIMAL(38, 0) AS (CAST(GET_PATH(PARSE_JSON(metadata$external_table_partition), 'COL3') AS DECIMAL(38, 0)))) PARTITION BY (col1, col2, col3) LOCATION=@s2/logs/ partition_type=user_specified FILE_FORMAT=(type=parquet compression=gzip binary_as_text=FALSE)",
) )
self.validate_all( self.validate_all(
@ -2092,6 +2110,15 @@ MATCH_RECOGNIZE (
self.assertEqual(ast.this, "DATABASES") self.assertEqual(ast.this, "DATABASES")
self.assertEqual(ast.args.get("scope_kind"), "ACCOUNT") self.assertEqual(ast.args.get("scope_kind"), "ACCOUNT")
def test_show_file_formats(self):
self.validate_identity("SHOW FILE FORMATS")
self.validate_identity("SHOW FILE FORMATS LIKE 'foo' IN DATABASE db1")
self.validate_identity("SHOW FILE FORMATS LIKE 'foo' IN SCHEMA db1.schema1")
ast = parse_one("SHOW FILE FORMATS IN ACCOUNT", read="snowflake")
self.assertEqual(ast.this, "FILE FORMATS")
self.assertEqual(ast.args.get("scope_kind"), "ACCOUNT")
def test_show_functions(self): def test_show_functions(self):
self.validate_identity("SHOW FUNCTIONS") self.validate_identity("SHOW FUNCTIONS")
self.validate_identity("SHOW FUNCTIONS LIKE 'foo' IN CLASS bla") self.validate_identity("SHOW FUNCTIONS LIKE 'foo' IN CLASS bla")
@ -2109,6 +2136,15 @@ MATCH_RECOGNIZE (
self.assertEqual(ast.this, "PROCEDURES") self.assertEqual(ast.this, "PROCEDURES")
self.assertEqual(ast.args.get("scope_kind"), "ACCOUNT") self.assertEqual(ast.args.get("scope_kind"), "ACCOUNT")
def test_show_stages(self):
self.validate_identity("SHOW STAGES")
self.validate_identity("SHOW STAGES LIKE 'foo' IN DATABASE db1")
self.validate_identity("SHOW STAGES LIKE 'foo' IN SCHEMA db1.schema1")
ast = parse_one("SHOW STAGES IN ACCOUNT", read="snowflake")
self.assertEqual(ast.this, "STAGES")
self.assertEqual(ast.args.get("scope_kind"), "ACCOUNT")
def test_show_warehouses(self): def test_show_warehouses(self):
self.validate_identity("SHOW WAREHOUSES") self.validate_identity("SHOW WAREHOUSES")
self.validate_identity("SHOW WAREHOUSES LIKE 'foo' WITH PRIVILEGES USAGE, MODIFY") self.validate_identity("SHOW WAREHOUSES LIKE 'foo' WITH PRIVILEGES USAGE, MODIFY")

10
tests/dialects/test_spark.py
View file

@ -245,6 +245,7 @@ TBLPROPERTIES (
"REFRESH TABLE t", "REFRESH TABLE t",
) )
self.validate_identity("IF(cond, foo AS bar, bla AS baz)")
self.validate_identity("any_value(col, true)", "ANY_VALUE(col) IGNORE NULLS") self.validate_identity("any_value(col, true)", "ANY_VALUE(col) IGNORE NULLS")
self.validate_identity("first(col, true)", "FIRST(col) IGNORE NULLS") self.validate_identity("first(col, true)", "FIRST(col) IGNORE NULLS")
self.validate_identity("first_value(col, true)", "FIRST_VALUE(col) IGNORE NULLS") self.validate_identity("first_value(col, true)", "FIRST_VALUE(col) IGNORE NULLS")
@ -307,6 +308,15 @@ TBLPROPERTIES (
"SELECT STR_TO_MAP('a:1,b:2,c:3')", "SELECT STR_TO_MAP('a:1,b:2,c:3')",
"SELECT STR_TO_MAP('a:1,b:2,c:3', ',', ':')", "SELECT STR_TO_MAP('a:1,b:2,c:3', ',', ':')",
) )
self.validate_all(
"SELECT TO_JSON(STRUCT('blah' AS x)) AS y",
write={
"presto": "SELECT JSON_FORMAT(CAST(CAST(ROW('blah') AS ROW(x VARCHAR)) AS JSON)) AS y",
"spark": "SELECT TO_JSON(STRUCT('blah' AS x)) AS y",
"trino": "SELECT JSON_FORMAT(CAST(CAST(ROW('blah') AS ROW(x VARCHAR)) AS JSON)) AS y",
},
)
self.validate_all( self.validate_all(
"SELECT TRY_ELEMENT_AT(ARRAY(1, 2, 3), 2)", "SELECT TRY_ELEMENT_AT(ARRAY(1, 2, 3), 2)",
read={ read={

4
tests/dialects/test_sqlite.py
View file

@ -109,6 +109,10 @@ class TestSQLite(Validator):
"SELECT * FROM station WHERE NOT city IS ''", "SELECT * FROM station WHERE NOT city IS ''",
) )
self.validate_identity("SELECT JSON_OBJECT('col1', 1, 'col2', '1')") self.validate_identity("SELECT JSON_OBJECT('col1', 1, 'col2', '1')")
self.validate_identity(
'CREATE TABLE "foo t" ("foo t id" TEXT NOT NULL, PRIMARY KEY ("foo t id"))',
'CREATE TABLE "foo t" ("foo t id" TEXT NOT NULL PRIMARY KEY)',
)
def test_strftime(self): def test_strftime(self):
self.validate_identity("SELECT STRFTIME('%Y/%m/%d', 'now')") self.validate_identity("SELECT STRFTIME('%Y/%m/%d', 'now')")

10
tests/dialects/test_tsql.py
View file

@ -17,6 +17,8 @@ class TestTSQL(Validator):
# tsql allows .. which means use the default schema # tsql allows .. which means use the default schema
self.validate_identity("SELECT * FROM a..b") self.validate_identity("SELECT * FROM a..b")
self.validate_identity("GO").assert_is(exp.Command)
self.validate_identity("SELECT go").selects[0].assert_is(exp.Column)
self.validate_identity("CREATE view a.b.c", "CREATE VIEW b.c") self.validate_identity("CREATE view a.b.c", "CREATE VIEW b.c")
self.validate_identity("DROP view a.b.c", "DROP VIEW b.c") self.validate_identity("DROP view a.b.c", "DROP VIEW b.c")
self.validate_identity("ROUND(x, 1, 0)") self.validate_identity("ROUND(x, 1, 0)")
@ -559,6 +561,14 @@ class TestTSQL(Validator):
with self.assertRaises(ParseError, msg=f"When running '{query}'"): with self.assertRaises(ParseError, msg=f"When running '{query}'"):
self.parse_one(query) self.parse_one(query)
self.validate_all(
"SELECT col FROM t OPTION(LABEL = 'foo')",
write={
"tsql": "SELECT col FROM t OPTION(LABEL = 'foo')",
"databricks": UnsupportedError,
},
)
def test_types(self): def test_types(self):
self.validate_identity("CAST(x AS XML)") self.validate_identity("CAST(x AS XML)")
self.validate_identity("CAST(x AS UNIQUEIDENTIFIER)") self.validate_identity("CAST(x AS UNIQUEIDENTIFIER)")

18
tests/fixtures/optimizer/merge_subqueries.sql vendored
View file

@ -481,3 +481,21 @@ FROM (
LEFT OUTER JOIN tbl AS ITBL ON OTBL.id = ITBL.id LEFT OUTER JOIN tbl AS ITBL ON OTBL.id = ITBL.id
) AS ITBL; ) AS ITBL;
WITH tbl AS (SELECT 1 AS id) SELECT OTBL.id AS id FROM tbl AS OTBL LEFT OUTER JOIN tbl AS ITBL_2 ON OTBL.id = ITBL_2.id LEFT OUTER JOIN tbl AS ITBL_3 ON OTBL.id = ITBL_3.id LEFT OUTER JOIN tbl AS ITBL ON OTBL.id = ITBL.id; WITH tbl AS (SELECT 1 AS id) SELECT OTBL.id AS id FROM tbl AS OTBL LEFT OUTER JOIN tbl AS ITBL_2 ON OTBL.id = ITBL_2.id LEFT OUTER JOIN tbl AS ITBL_3 ON OTBL.id = ITBL_3.id LEFT OUTER JOIN tbl AS ITBL ON OTBL.id = ITBL.id;
# title: Inner query contains subquery with an alias that conflicts with outer query
WITH i AS (
SELECT
a
FROM (
SELECT 1 a
) AS conflict
), j AS (
SELECT 1 AS a
)
SELECT
i.a,
conflict.a
FROM i
LEFT JOIN j AS conflict
ON i.a = conflict.a;
WITH j AS (SELECT 1 AS a) SELECT conflict_2.a AS a, conflict.a AS a FROM (SELECT 1 AS a) AS conflict_2 LEFT JOIN j AS conflict ON conflict_2.a = conflict.a;

6
tests/fixtures/optimizer/optimizer.sql vendored
View file

@ -760,10 +760,12 @@ SELECT
`_q_0`.`first_half_sales` AS `first_half_sales`, `_q_0`.`first_half_sales` AS `first_half_sales`,
`_q_0`.`second_half_sales` AS `second_half_sales` `_q_0`.`second_half_sales` AS `second_half_sales`
FROM `produce` AS `produce` FROM `produce` AS `produce`
UNPIVOT((`first_half_sales`, `second_half_sales`) FOR `semesters` IN ( UNPIVOT((`first_half_sales`, `second_half_sales`) FOR
`semesters` IN (
(`produce`.`q1`, `produce`.`q2`) AS 'semester_1', (`produce`.`q1`, `produce`.`q2`) AS 'semester_1',
(`produce`.`q3`, `produce`.`q4`) AS 'semester_2' (`produce`.`q3`, `produce`.`q4`) AS 'semester_2'
)) AS `_q_0`; )
) AS `_q_0`;
# title: quoting is preserved # title: quoting is preserved
# dialect: snowflake # dialect: snowflake

82
tests/fixtures/optimizer/qualify_columns.sql vendored
View file

@ -325,6 +325,65 @@ SELECT _q_0.a AS a FROM (SELECT x.a AS a FROM x AS x UNION SELECT x.a AS a FROM
((select a from x where a < 1)) UNION ((select a from x where a > 2)); ((select a from x where a < 1)) UNION ((select a from x where a > 2));
((SELECT x.a AS a FROM x AS x WHERE x.a < 1)) UNION ((SELECT x.a AS a FROM x AS x WHERE x.a > 2)); ((SELECT x.a AS a FROM x AS x WHERE x.a < 1)) UNION ((SELECT x.a AS a FROM x AS x WHERE x.a > 2));
# dialect: bigquery
# execute: false
SELECT * FROM (SELECT 1 AS foo, 2 AS bar INNER UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz);
SELECT _q_0.bar AS bar FROM (SELECT 1 AS foo, 2 AS bar INNER UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz) AS _q_0;
# dialect: bigquery
# execute: false
SELECT * FROM (SELECT 1 AS foo, 2 AS bar UNION ALL CORRESPONDING SELECT 3 AS bar, 4 AS baz);
SELECT _q_0.bar AS bar FROM (SELECT 1 AS foo, 2 AS bar INNER UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz) AS _q_0;
# dialect: bigquery
# execute: false
SELECT * FROM (SELECT 1 AS foo, 2 AS bar LEFT UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz);
SELECT _q_0.foo AS foo, _q_0.bar AS bar FROM (SELECT 1 AS foo, 2 AS bar LEFT UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz) AS _q_0;
# dialect: bigquery
# execute: false
SELECT * FROM (SELECT 1 AS foo, 2 AS bar FULL UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz);
SELECT _q_0.foo AS foo, _q_0.bar AS bar, _q_0.baz AS baz FROM (SELECT 1 AS foo, 2 AS bar FULL UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz) AS _q_0;
# dialect: bigquery
# execute: false
SELECT * FROM (SELECT 1 AS foo, 2 AS bar LEFT UNION ALL CORRESPONDING SELECT 3 AS bar, 4 AS baz);
SELECT _q_0.foo AS foo, _q_0.bar AS bar FROM (SELECT 1 AS foo, 2 AS bar LEFT UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz) AS _q_0;
# dialect: bigquery
# execute: false
SELECT * FROM (SELECT 1 AS foo, 2 AS bar FULL UNION ALL CORRESPONDING SELECT 3 AS bar, 4 AS baz);
SELECT _q_0.foo AS foo, _q_0.bar AS bar, _q_0.baz AS baz FROM (SELECT 1 AS foo, 2 AS bar FULL UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz) AS _q_0;
# dialect: bigquery
# execute: false
SELECT * FROM (SELECT 1 AS foo, 2 AS bar FULL UNION ALL CORRESPONDING BY (foo, bar) SELECT 3 AS bar, 4 AS baz);
SELECT _q_0.foo AS foo, _q_0.bar AS bar FROM (SELECT 1 AS foo, 2 AS bar FULL UNION ALL BY NAME ON (foo, bar) SELECT 3 AS bar, 4 AS baz) AS _q_0;
# dialect: bigquery
# execute: false
SELECT * FROM (SELECT 1 AS foo, 2 AS bar FULL UNION ALL BY NAME ON (foo, bar) SELECT 3 AS bar, 4 AS baz);
SELECT _q_0.foo AS foo, _q_0.bar AS bar FROM (SELECT 1 AS foo, 2 AS bar FULL UNION ALL BY NAME ON (foo, bar) SELECT 3 AS bar, 4 AS baz) AS _q_0;
# dialect: bigquery
# execute: false
SELECT * FROM ((SELECT 1 AS foo, 2 AS bar LEFT UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz) LEFT UNION ALL BY NAME ON (bar) SELECT 3 AS foo, 4 AS bar);
SELECT _q_0.bar AS bar FROM ((SELECT 1 AS foo, 2 AS bar LEFT UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz) LEFT UNION ALL BY NAME ON (bar) SELECT 3 AS foo, 4 AS bar) AS _q_0;
# dialect: bigquery
# execute: false
SELECT * FROM ((SELECT 1 AS foo, 2 AS bar LEFT UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz) FULL UNION ALL BY NAME ON (foo, qux) SELECT 3 AS qux, 4 AS bar);
SELECT _q_0.foo AS foo, _q_0.qux AS qux FROM ((SELECT 1 AS foo, 2 AS bar LEFT UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz) FULL UNION ALL BY NAME ON (foo, qux) SELECT 3 AS qux, 4 AS bar) AS _q_0;
# dialect: bigquery
# execute: false
SELECT * FROM (((SELECT 1 AS foo, 2 AS bar LEFT UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz) FULL UNION ALL BY NAME ON (foo, qux) SELECT 3 AS qux, 4 AS bar) INNER UNION ALL BY NAME ON (foo) SELECT 6 AS foo);
SELECT _q_0.foo AS foo FROM (((SELECT 1 AS foo, 2 AS bar LEFT UNION ALL BY NAME SELECT 3 AS bar, 4 AS baz) FULL UNION ALL BY NAME ON (foo, qux) SELECT 3 AS qux, 4 AS bar) INNER UNION ALL BY NAME ON (foo) SELECT 6 AS foo) AS _q_0;
-------------------------------------- --------------------------------------
-- Subqueries -- Subqueries
-------------------------------------- --------------------------------------
@ -435,6 +494,29 @@ WITH tbl1 AS (SELECT STRUCT(1 AS col1, 2 AS col1) AS col) SELECT tbl1.col.* FROM
SELECT * FROM READ_CSV('file.csv'); SELECT * FROM READ_CSV('file.csv');
SELECT * FROM READ_CSV('file.csv') AS _q_0; SELECT * FROM READ_CSV('file.csv') AS _q_0;
# dialect: clickhouse
# Title: Expand tuples in VALUES using the structure provided
# execute: false
SELECT * FROM VALUES ('person String, place String', ('Noah', 'Paris'));
SELECT _q_0.person AS person, _q_0.place AS place FROM VALUES ('person String, place String', ('Noah', 'Paris')) AS _q_0(person, place);
# dialect: clickhouse
# Title: Expand tuples in VALUES using the default naming scheme in CH
# execute: false
SELECT * FROM VALUES ((1, 1), (2, 2));
SELECT _q_0.c1 AS c1, _q_0.c2 AS c2 FROM VALUES ((1, 1), (2, 2)) AS _q_0(c1, c2);
# dialect: clickhouse
# Title: Expand fields in VALUES using the default naming scheme in CH
# execute: false
SELECT * FROM VALUES (1, 2, 3);
SELECT _q_0.c1 AS c1 FROM VALUES ((1), (2), (3)) AS _q_0(c1);
# title: Expand PIVOT column combinations
# dialect: duckdb
WITH cities AS (SELECT * FROM (VALUES ('nl', 'amsterdam', 2000, 1005)) AS t(country, name, year, population)) SELECT * FROM cities PIVOT(SUM(population) AS total, COUNT(population) AS count FOR country IN ('nl', 'us') year IN (2000, 2010) name IN ('amsterdam', 'seattle'));
WITH cities AS (SELECT t.country AS country, t.name AS name, t.year AS year, t.population AS population FROM (VALUES ('nl', 'amsterdam', 2000, 1005)) AS t(country, name, year, population)) SELECT _q_0.nl_2000_amsterdam_total AS nl_2000_amsterdam_total, _q_0.nl_2000_amsterdam_count AS nl_2000_amsterdam_count, _q_0.nl_2000_seattle_total AS nl_2000_seattle_total, _q_0.nl_2000_seattle_count AS nl_2000_seattle_count, _q_0.nl_2010_amsterdam_total AS nl_2010_amsterdam_total, _q_0.nl_2010_amsterdam_count AS nl_2010_amsterdam_count, _q_0.nl_2010_seattle_total AS nl_2010_seattle_total, _q_0.nl_2010_seattle_count AS nl_2010_seattle_count, _q_0.us_2000_amsterdam_total AS us_2000_amsterdam_total, _q_0.us_2000_amsterdam_count AS us_2000_amsterdam_count, _q_0.us_2000_seattle_total AS us_2000_seattle_total, _q_0.us_2000_seattle_count AS us_2000_seattle_count, _q_0.us_2010_amsterdam_total AS us_2010_amsterdam_total, _q_0.us_2010_amsterdam_count AS us_2010_amsterdam_count, _q_0.us_2010_seattle_total AS us_2010_seattle_total, _q_0.us_2010_seattle_count AS us_2010_seattle_count FROM cities AS cities PIVOT(SUM(population) AS total, COUNT(population) AS count FOR country IN ('nl', 'us') year IN (2000, 2010) name IN ('amsterdam', 'seattle')) AS _q_0;
-------------------------------------- --------------------------------------
-- CTEs -- CTEs
-------------------------------------- --------------------------------------

5
tests/test_optimizer.py
View file

@ -55,13 +55,10 @@ def simplify(expression, **kwargs):
def annotate_functions(expression, **kwargs): def annotate_functions(expression, **kwargs):
from sqlglot.dialects import Dialect
dialect = kwargs.get("dialect") dialect = kwargs.get("dialect")
schema = kwargs.get("schema") schema = kwargs.get("schema")
annotators = Dialect.get_or_raise(dialect).ANNOTATORS annotated = annotate_types(expression, dialect=dialect, schema=schema)
annotated = annotate_types(expression, annotators=annotators, schema=schema)
return annotated.expressions[0] return annotated.expressions[0]

58
tests/test_parser.py
View file

@ -646,6 +646,27 @@ class TestParser(unittest.TestCase):
) PIVOT (AVG("PrIcE"), MAX(quality) FOR partname IN ('prop' AS prop1, 'rudder')) ) PIVOT (AVG("PrIcE"), MAX(quality) FOR partname IN ('prop' AS prop1, 'rudder'))
""" """
two_in_clauses_duckdb = """
SELECT * FROM cities PIVOT (
sum(population) AS total,
count(population) AS count
FOR
year IN (2000, 2010)
country IN ('NL', 'US')
)
"""
three_in_clauses_duckdb = """
SELECT * FROM cities PIVOT (
sum(population) AS total,
count(population) AS count
FOR
year IN (2000, 2010)
country IN ('NL', 'US')
name IN ('Amsterdam', 'Seattle')
)
"""
query_to_column_names = { query_to_column_names = {
nothing_aliased: { nothing_aliased: {
"bigquery": ["prop", "rudder"], "bigquery": ["prop", "rudder"],
@ -707,13 +728,48 @@ class TestParser(unittest.TestCase):
'"rudder_max(quality)"', '"rudder_max(quality)"',
], ],
}, },
two_in_clauses_duckdb: {
"duckdb": [
'"2000_NL_total"',
'"2000_NL_count"',
'"2000_US_total"',
'"2000_US_count"',
'"2010_NL_total"',
'"2010_NL_count"',
'"2010_US_total"',
'"2010_US_count"',
],
},
three_in_clauses_duckdb: {
"duckdb": [
'"2000_NL_Amsterdam_total"',
'"2000_NL_Amsterdam_count"',
'"2000_NL_Seattle_total"',
'"2000_NL_Seattle_count"',
'"2000_US_Amsterdam_total"',
'"2000_US_Amsterdam_count"',
'"2000_US_Seattle_total"',
'"2000_US_Seattle_count"',
'"2010_NL_Amsterdam_total"',
'"2010_NL_Amsterdam_count"',
'"2010_NL_Seattle_total"',
'"2010_NL_Seattle_count"',
'"2010_US_Amsterdam_total"',
'"2010_US_Amsterdam_count"',
'"2010_US_Seattle_total"',
'"2010_US_Seattle_count"',
],
},
} }
for query, dialect_columns in query_to_column_names.items(): for query, dialect_columns in query_to_column_names.items():
for dialect, expected_columns in dialect_columns.items(): for dialect, expected_columns in dialect_columns.items():
with self.subTest(f"Testing query '{query}' for dialect {dialect}"):
expr = parse_one(query, read=dialect) expr = parse_one(query, read=dialect)
columns = expr.args["from"].this.args["pivots"][0].args["columns"] columns = expr.args["from"].this.args["pivots"][0].args["columns"]
self.assertEqual(expected_columns, [col.sql(dialect=dialect) for col in columns]) self.assertEqual(
expected_columns, [col.sql(dialect=dialect) for col in columns]
)
def test_parse_nested(self): def test_parse_nested(self):
def warn_over_threshold(query: str, max_threshold: float = 0.2): def warn_over_threshold(query: str, max_threshold: float = 0.2):