1
0
Fork 0

Merging upstream version 25.16.1.

Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-13 21:52:32 +01:00
parent 7688e2bdf8
commit bad79d1f7c
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
110 changed files with 75353 additions and 68092 deletions

View file

@ -1,6 +1,247 @@
Changelog
=========
## [v25.16.0] - 2024-08-22
### :boom: BREAKING CHANGES
- due to [`f68d155`](https://github.com/tobymao/sqlglot/commit/f68d155c38a79a6527685c37f8de8773ce790bca) - exp.Merge, for Trino and Postgres, dont strip the target alias from then WHEN MATCHED condition to prevent an ambiguous column error *(PR [#3940](https://github.com/tobymao/sqlglot/pull/3940) by [@erindru](https://github.com/erindru))*:
exp.Merge, for Trino and Postgres, dont strip the target alias from then WHEN MATCHED condition to prevent an ambiguous column error (#3940)
- due to [`667f7d9`](https://github.com/tobymao/sqlglot/commit/667f7d9e94e14ff619998d2001b6116d363f2a1f) - attach INTERPOLATE expressions to WithFill *(PR [#3944](https://github.com/tobymao/sqlglot/pull/3944) by [@georgesittas](https://github.com/georgesittas))*:
attach INTERPOLATE expressions to WithFill (#3944)
- due to [`145fdbf`](https://github.com/tobymao/sqlglot/commit/145fdbf6bb02fa1c55087bfd9f6b3a15fbd4b684) - Redshift date format *(PR [#3942](https://github.com/tobymao/sqlglot/pull/3942) by [@erindru](https://github.com/erindru))*:
Redshift date format (#3942)
- due to [`a84a21a`](https://github.com/tobymao/sqlglot/commit/a84a21aaef0e65754e67ecebdfcbf7136c77acc7) - Add timezone support to exp.TimeStrToTime *(PR [#3938](https://github.com/tobymao/sqlglot/pull/3938) by [@erindru](https://github.com/erindru))*:
Add timezone support to exp.TimeStrToTime (#3938)
### :sparkles: New Features
- [`a84a21a`](https://github.com/tobymao/sqlglot/commit/a84a21aaef0e65754e67ecebdfcbf7136c77acc7) - Add timezone support to exp.TimeStrToTime *(PR [#3938](https://github.com/tobymao/sqlglot/pull/3938) by [@erindru](https://github.com/erindru))*
- [`70a052a`](https://github.com/tobymao/sqlglot/commit/70a052a672d0c72a3e53b19316defb01144f2907) - transpile from_iso8601_timestamp from presto/trino to duckdb *(PR [#3956](https://github.com/tobymao/sqlglot/pull/3956) by [@georgesittas](https://github.com/georgesittas))*
### :bug: Bug Fixes
- [`f68d155`](https://github.com/tobymao/sqlglot/commit/f68d155c38a79a6527685c37f8de8773ce790bca) - exp.Merge, for Trino and Postgres, dont strip the target alias from then WHEN MATCHED condition to prevent an ambiguous column error *(PR [#3940](https://github.com/tobymao/sqlglot/pull/3940) by [@erindru](https://github.com/erindru))*
- [`0458dc0`](https://github.com/tobymao/sqlglot/commit/0458dc0fa1978388336b9fa459b28508d7b40f9e) - **optimizer**: expand alias refs recursive CTE edge case patch *(PR [#3943](https://github.com/tobymao/sqlglot/pull/3943) by [@georgesittas](https://github.com/georgesittas))*
- [`145fdbf`](https://github.com/tobymao/sqlglot/commit/145fdbf6bb02fa1c55087bfd9f6b3a15fbd4b684) - Redshift date format *(PR [#3942](https://github.com/tobymao/sqlglot/pull/3942) by [@erindru](https://github.com/erindru))*
- [`6233c2c`](https://github.com/tobymao/sqlglot/commit/6233c2c75ab3a3bc0dfbf28d3fa8adc1be719281) - **parser**: Support sqls with DESCRIBE partition *(PR [#3945](https://github.com/tobymao/sqlglot/pull/3945) by [@gp1105739](https://github.com/gp1105739))*
- :arrow_lower_right: *fixes issue [#3941](https://github.com/tobymao/sqlglot/issues/3941) opened by [@gp1105739](https://github.com/gp1105739)*
- [`85cd6e5`](https://github.com/tobymao/sqlglot/commit/85cd6e507b73be89d2d9b2c88c7370a14b813b5c) - **bigquery**: Map %e to %-d *(PR [#3946](https://github.com/tobymao/sqlglot/pull/3946) by [@VaggelisD](https://github.com/VaggelisD))*
- [`1ba0f03`](https://github.com/tobymao/sqlglot/commit/1ba0f03fbfe5dadc3411c7ff26e6dfbef852491a) - **duckdb**: TIME does not support modifiers *(PR [#3947](https://github.com/tobymao/sqlglot/pull/3947) by [@georgesittas](https://github.com/georgesittas))*
- [`d5d3615`](https://github.com/tobymao/sqlglot/commit/d5d361571cd463869e2243d257f9b6ad0615c070) - **optimizer**: convert TsOrDsToDate to Cast more conservatively *(PR [#3949](https://github.com/tobymao/sqlglot/pull/3949) by [@barakalon](https://github.com/barakalon))*
- [`fb6edc7`](https://github.com/tobymao/sqlglot/commit/fb6edc774539704b48e7d2805ef3211636af18aa) - oracle/snowflake comments closes [#3950](https://github.com/tobymao/sqlglot/pull/3950) *(commit by [@tobymao](https://github.com/tobymao))*
- [`1284fd0`](https://github.com/tobymao/sqlglot/commit/1284fd0a64890d3548af7ed0a0cc05bb6166ccb2) - **oracle**: Revert NVL() being parsed into exp.Anonymous *(PR [#3954](https://github.com/tobymao/sqlglot/pull/3954) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#3952](https://github.com/tobymao/sqlglot/issues/3952) opened by [@sleshJdev](https://github.com/sleshJdev)*
- [`c99f8d5`](https://github.com/tobymao/sqlglot/commit/c99f8d5bda79f16fb0d71ae73127cc826860e104) - **duckdb**: Fix exp.Unnest generation for BQ's nested arrays *(PR [#3931](https://github.com/tobymao/sqlglot/pull/3931) by [@VaggelisD](https://github.com/VaggelisD))*
### :recycle: Refactors
- [`f16b0e7`](https://github.com/tobymao/sqlglot/commit/f16b0e7203ad60f0ce50861c4d78176ca53eb2cf) - iteratively generate binary expressions *(PR [#3926](https://github.com/tobymao/sqlglot/pull/3926) by [@MatMoore](https://github.com/MatMoore))*
- [`667f7d9`](https://github.com/tobymao/sqlglot/commit/667f7d9e94e14ff619998d2001b6116d363f2a1f) - **clickhouse**: attach INTERPOLATE expressions to WithFill *(PR [#3944](https://github.com/tobymao/sqlglot/pull/3944) by [@georgesittas](https://github.com/georgesittas))*
### :wrench: Chores
- [`c697357`](https://github.com/tobymao/sqlglot/commit/c6973572dfd953b5539bb4e9dcba402c0c3c6acf) - slightly refactor Generator.binary, add stress test *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`6a5f619`](https://github.com/tobymao/sqlglot/commit/6a5f6199f6da0053fa4564e71a17e3b9f91f0496) - New doc - Onboarding Doc *(PR [#3902](https://github.com/tobymao/sqlglot/pull/3902) by [@VaggelisD](https://github.com/VaggelisD))*
## [v25.15.0] - 2024-08-19
### :boom: BREAKING CHANGES
- due to [`a668655`](https://github.com/tobymao/sqlglot/commit/a668655440815605a566c52b65b28decdfb551eb) - preserve SYSDATE *(PR [#3935](https://github.com/tobymao/sqlglot/pull/3935) by [@georgesittas](https://github.com/georgesittas))*:
preserve SYSDATE (#3935)
### :sparkles: New Features
- [`be11f4c`](https://github.com/tobymao/sqlglot/commit/be11f4c57c7842f69950bafc3225fb9c139af014) - **clickhouse**: add support for "@"-style parameters *(PR [#3939](https://github.com/tobymao/sqlglot/pull/3939) by [@georgesittas](https://github.com/georgesittas))*
### :bug: Bug Fixes
- [`a668655`](https://github.com/tobymao/sqlglot/commit/a668655440815605a566c52b65b28decdfb551eb) - **oracle**: preserve SYSDATE *(PR [#3935](https://github.com/tobymao/sqlglot/pull/3935) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3934](https://github.com/tobymao/sqlglot/issues/3934) opened by [@Hal-H2Apps](https://github.com/Hal-H2Apps)*
- [`b824f8a`](https://github.com/tobymao/sqlglot/commit/b824f8a4148ace01750db301daf4a663dc03b580) - **parser**: allow complex expressions for UNPIVOT alias *(PR [#3937](https://github.com/tobymao/sqlglot/pull/3937) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3936](https://github.com/tobymao/sqlglot/issues/3936) opened by [@dbittenbender](https://github.com/dbittenbender)*
### :recycle: Refactors
- [`f4c34d3`](https://github.com/tobymao/sqlglot/commit/f4c34d37c5773c37a13437c7e0e7eb27b4e98877) - move "MINUS": TokenType.EXCEPT to hive instead of spark *(commit by [@georgesittas](https://github.com/georgesittas))*
## [v25.14.0] - 2024-08-19
### :boom: BREAKING CHANGES
- due to [`605f1b2`](https://github.com/tobymao/sqlglot/commit/605f1b217d5d1de654cfe2fa1b51435a1a71ae62) - use creatable kind mapping dict for schema<-->database substitution *(PR [#3924](https://github.com/tobymao/sqlglot/pull/3924) by [@treysp](https://github.com/treysp))*:
use creatable kind mapping dict for schema<-->database substitution (#3924)
- due to [`f418caa`](https://github.com/tobymao/sqlglot/commit/f418caafa8ed317f9e360c6c8f01bdac596258e5) - skip nullable comparison in is_type by default *(PR [#3927](https://github.com/tobymao/sqlglot/pull/3927) by [@georgesittas](https://github.com/georgesittas))*:
skip nullable comparison in is_type by default (#3927)
### :sparkles: New Features
- [`f418caa`](https://github.com/tobymao/sqlglot/commit/f418caafa8ed317f9e360c6c8f01bdac596258e5) - skip nullable comparison in is_type by default *(PR [#3927](https://github.com/tobymao/sqlglot/pull/3927) by [@georgesittas](https://github.com/georgesittas))*
### :bug: Bug Fixes
- [`605f1b2`](https://github.com/tobymao/sqlglot/commit/605f1b217d5d1de654cfe2fa1b51435a1a71ae62) - **clickhouse**: use creatable kind mapping dict for schema<-->database substitution *(PR [#3924](https://github.com/tobymao/sqlglot/pull/3924) by [@treysp](https://github.com/treysp))*
## [v25.13.0] - 2024-08-17
### :boom: BREAKING CHANGES
- due to [`102f5d4`](https://github.com/tobymao/sqlglot/commit/102f5d48279ac1a7a1851737f55a13bd08512f3d) - infer set op types more accurately *(PR [#3918](https://github.com/tobymao/sqlglot/pull/3918) by [@georgesittas](https://github.com/georgesittas))*:
infer set op types more accurately (#3918)
- due to [`46496a6`](https://github.com/tobymao/sqlglot/commit/46496a6af80bd49d36ef8d265800679d2b07c4db) - improve transpilation of nullable/non-nullable data types *(PR [#3921](https://github.com/tobymao/sqlglot/pull/3921) by [@georgesittas](https://github.com/georgesittas))*:
improve transpilation of nullable/non-nullable data types (#3921)
### :bug: Bug Fixes
- [`c74a8fd`](https://github.com/tobymao/sqlglot/commit/c74a8fd2acd859f5947f27a8f091f13fba1d39e4) - **clickhouse**: make try_cast toXXXOrNull() functions case-specific *(PR [#3917](https://github.com/tobymao/sqlglot/pull/3917) by [@treysp](https://github.com/treysp))*
- [`102f5d4`](https://github.com/tobymao/sqlglot/commit/102f5d48279ac1a7a1851737f55a13bd08512f3d) - **optimizer**: infer set op types more accurately *(PR [#3918](https://github.com/tobymao/sqlglot/pull/3918) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3916](https://github.com/tobymao/sqlglot/issues/3916) opened by [@racevedoo](https://github.com/racevedoo)*
### :recycle: Refactors
- [`1d436d4`](https://github.com/tobymao/sqlglot/commit/1d436d45b4469bb8195dd3597319b6fc5c3f2344) - **clickhouse**: transpile TRY_CAST(x AS T) to CAST(x AS Nullable(T)) *(PR [#3919](https://github.com/tobymao/sqlglot/pull/3919) by [@georgesittas](https://github.com/georgesittas))*
- [`46496a6`](https://github.com/tobymao/sqlglot/commit/46496a6af80bd49d36ef8d265800679d2b07c4db) - **clickhouse**: improve transpilation of nullable/non-nullable data types *(PR [#3921](https://github.com/tobymao/sqlglot/pull/3921) by [@georgesittas](https://github.com/georgesittas))*
## [v25.12.0] - 2024-08-15
### :boom: BREAKING CHANGES
- due to [`e8e70f3`](https://github.com/tobymao/sqlglot/commit/e8e70f3a6cc2ca24de2afe622bbcbccb1ac8aeb3) - treat DATABASE kind as SCHEMA (and conversely) in exp.Create *(PR [#3912](https://github.com/tobymao/sqlglot/pull/3912) by [@georgesittas](https://github.com/georgesittas))*:
treat DATABASE kind as SCHEMA (and conversely) in exp.Create (#3912)
### :sparkles: New Features
- [`9a66903`](https://github.com/tobymao/sqlglot/commit/9a66903975f16a09d84337a8405bf70945706412) - **clickhouse**: add support for TryCast generation *(PR [#3913](https://github.com/tobymao/sqlglot/pull/3913) by [@georgesittas](https://github.com/georgesittas))*
### :bug: Bug Fixes
- [`7965cac`](https://github.com/tobymao/sqlglot/commit/7965cace1d9632c865cae257781072b0932b709d) - **clickhouse**: wrap query in CTAS when COMMENT prop is present *(PR [#3911](https://github.com/tobymao/sqlglot/pull/3911) by [@georgesittas](https://github.com/georgesittas))*
- [`e8e70f3`](https://github.com/tobymao/sqlglot/commit/e8e70f3a6cc2ca24de2afe622bbcbccb1ac8aeb3) - **clickhouse**: treat DATABASE kind as SCHEMA (and conversely) in exp.Create *(PR [#3912](https://github.com/tobymao/sqlglot/pull/3912) by [@georgesittas](https://github.com/georgesittas))*
## [v25.11.3] - 2024-08-14
### :bug: Bug Fixes
- [`57f7aa9`](https://github.com/tobymao/sqlglot/commit/57f7aa9108ed38c0e83ef5bf4fac900434fac777) - **clickhouse**: COMMENT property in CTAS needs to come last *(PR [#3910](https://github.com/tobymao/sqlglot/pull/3910) by [@georgesittas](https://github.com/georgesittas))*
## [v25.11.2] - 2024-08-14
### :bug: Bug Fixes
- [`c22f411`](https://github.com/tobymao/sqlglot/commit/c22f41129985ecfd3b3906b9594ca1692b91708c) - **clickhouse**: ensure we generate the Table in creatable_sql if it represents a db ref *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`19eee93`](https://github.com/tobymao/sqlglot/commit/19eee93c8027e6c612611d3b54980e193e0b6f49) - various fixups for unnest(generatedatearray) transpilation *(PR [#3906](https://github.com/tobymao/sqlglot/pull/3906) by [@georgesittas](https://github.com/georgesittas))*
## [v25.11.1] - 2024-08-13
### :sparkles: New Features
- [`790c1b1`](https://github.com/tobymao/sqlglot/commit/790c1b141d4bc2206df017c70416b589932886a4) - **clickhouse**: support PARTITION BY, SETTINGS in Insert expression *(PR [#3904](https://github.com/tobymao/sqlglot/pull/3904) by [@georgesittas](https://github.com/georgesittas))*
## [v25.11.0] - 2024-08-13
### :boom: BREAKING CHANGES
- due to [`0428c37`](https://github.com/tobymao/sqlglot/commit/0428c37e11f42be8eba352e69c1d2e7425824d38) - Support ALTER VIEW AS SELECT *(PR [#3873](https://github.com/tobymao/sqlglot/pull/3873) by [@xiaohui-sun](https://github.com/xiaohui-sun))*:
Support ALTER VIEW AS SELECT (#3873)
- due to [`a666117`](https://github.com/tobymao/sqlglot/commit/a666117dcb887031f5995c50d687405b9c145fbd) - parse v NOT IN (subquery) as v <> ALL (subquery) *(PR [#3891](https://github.com/tobymao/sqlglot/pull/3891) by [@georgesittas](https://github.com/georgesittas))*:
parse v NOT IN (subquery) as v <> ALL (subquery) (#3891)
- due to [`d968932`](https://github.com/tobymao/sqlglot/commit/d968932ef742e97ccf3ec6cdca0bc3319830f0a9) - treat identifiers as case-sensitive, handle EMPTY table property, generate DateStrToDate *(PR [#3895](https://github.com/tobymao/sqlglot/pull/3895) by [@jwhitaker-gridcog](https://github.com/jwhitaker-gridcog))*:
treat identifiers as case-sensitive, handle EMPTY table property, generate DateStrToDate (#3895)
- due to [`1d7319a`](https://github.com/tobymao/sqlglot/commit/1d7319a8425aace6c11f59552fdd19bdbf5efd03) - transpile Unnest(GenerateDateArray(...)) to various dialects *(PR [#3899](https://github.com/tobymao/sqlglot/pull/3899) by [@georgesittas](https://github.com/georgesittas))*:
transpile Unnest(GenerateDateArray(...)) to various dialects (#3899)
### :sparkles: New Features
- [`0428c37`](https://github.com/tobymao/sqlglot/commit/0428c37e11f42be8eba352e69c1d2e7425824d38) - **parser**: Support ALTER VIEW AS SELECT *(PR [#3873](https://github.com/tobymao/sqlglot/pull/3873) by [@xiaohui-sun](https://github.com/xiaohui-sun))*
- [`8a48458`](https://github.com/tobymao/sqlglot/commit/8a48458e20e6d0833638e750565da138bdcd5d55) - **athena**: parse UNLOAD into exp.Command closes [#3896](https://github.com/tobymao/sqlglot/pull/3896) *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`6f1527f`](https://github.com/tobymao/sqlglot/commit/6f1527fd3ebd16f49edb351f050a1db687824530) - **bigquery**: transpile format_datetime, datetime_trunc to duckdb *(PR [#3894](https://github.com/tobymao/sqlglot/pull/3894) by [@skadel](https://github.com/skadel))*
- [`1d7319a`](https://github.com/tobymao/sqlglot/commit/1d7319a8425aace6c11f59552fdd19bdbf5efd03) - transpile Unnest(GenerateDateArray(...)) to various dialects *(PR [#3899](https://github.com/tobymao/sqlglot/pull/3899) by [@georgesittas](https://github.com/georgesittas))*
### :bug: Bug Fixes
- [`2cac14f`](https://github.com/tobymao/sqlglot/commit/2cac14f480dcaf458b1eb36b694770ce24f56e61) - generate set ops in ALTER VIEW AS statement *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`a666117`](https://github.com/tobymao/sqlglot/commit/a666117dcb887031f5995c50d687405b9c145fbd) - **snowflake**: parse v NOT IN (subquery) as v <> ALL (subquery) *(PR [#3891](https://github.com/tobymao/sqlglot/pull/3891) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3890](https://github.com/tobymao/sqlglot/issues/3890) opened by [@ajuszczak](https://github.com/ajuszczak)*
- [`924a4af`](https://github.com/tobymao/sqlglot/commit/924a4af146952e84688fdccb7b63883fcd7fb255) - **oracle**: preserve function-style MOD syntax fixes [#3897](https://github.com/tobymao/sqlglot/pull/3897) *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`d968932`](https://github.com/tobymao/sqlglot/commit/d968932ef742e97ccf3ec6cdca0bc3319830f0a9) - **clickhouse**: treat identifiers as case-sensitive, handle EMPTY table property, generate DateStrToDate *(PR [#3895](https://github.com/tobymao/sqlglot/pull/3895) by [@jwhitaker-gridcog](https://github.com/jwhitaker-gridcog))*
- [`3e5e730`](https://github.com/tobymao/sqlglot/commit/3e5e7300ec184024f871669db48d68476b3fa4df) - **clickhouse**: generate exp.Values correctly, handle `FORMAT Values` *(PR [#3900](https://github.com/tobymao/sqlglot/pull/3900) by [@georgesittas](https://github.com/georgesittas))*
- [`bea3c08`](https://github.com/tobymao/sqlglot/commit/bea3c08e46a020d8545b702c77f0db18c99f1c55) - **parser**: improve performance of OUTER/CROSS APPLY parsing *(PR [#3901](https://github.com/tobymao/sqlglot/pull/3901) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3898](https://github.com/tobymao/sqlglot/issues/3898) opened by [@ewhitley](https://github.com/ewhitley)*
## [v25.10.0] - 2024-08-08
### :boom: BREAKING CHANGES
- due to [`3eb46db`](https://github.com/tobymao/sqlglot/commit/3eb46db5c429f50b5bb6c0c5517a5f7c1084b5ea) - switch off CSV file schema inference by default *(PR [#3879](https://github.com/tobymao/sqlglot/pull/3879) by [@georgesittas](https://github.com/georgesittas))*:
switch off CSV file schema inference by default (#3879)
### :sparkles: New Features
- [`3e4fcf7`](https://github.com/tobymao/sqlglot/commit/3e4fcf7e8f6a322c14470de6c5dbba152bc9b2fe) - **databricks**: Add support for STREAMING tables *(PR [#3878](https://github.com/tobymao/sqlglot/pull/3878) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *addresses issue [#3876](https://github.com/tobymao/sqlglot/issues/3876) opened by [@ericvergnaud](https://github.com/ericvergnaud)*
- [`528f690`](https://github.com/tobymao/sqlglot/commit/528f6908001db2f132edfa3c61c21815f7e9dc2f) - **duckdb**: Transpile Snowflake's CONVERT_TIMEZONE 3-arg version *(PR [#3883](https://github.com/tobymao/sqlglot/pull/3883) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *addresses issue [#3875](https://github.com/tobymao/sqlglot/issues/3875) opened by [@milonimrod](https://github.com/milonimrod)*
- [`411f62a`](https://github.com/tobymao/sqlglot/commit/411f62ad27f8cbe0d9a429e0cafdf4bd9eb2749f) - **bigquery**: Support for GENERATE_TIMESTAMP_ARRAY, DDB transpilation *(PR [#3888](https://github.com/tobymao/sqlglot/pull/3888) by [@VaggelisD](https://github.com/VaggelisD))*
### :bug: Bug Fixes
- [`7169e6e`](https://github.com/tobymao/sqlglot/commit/7169e6ef52d24754059b9ee4324398d22ddff0da) - **bigquery**: ensure Funcs are preserved when used as Tables *(PR [#3877](https://github.com/tobymao/sqlglot/pull/3877) by [@georgesittas](https://github.com/georgesittas))*
- [`62ceed2`](https://github.com/tobymao/sqlglot/commit/62ceed2fa3cd7b41919839d837b860f3814fa769) - **redshift**: parse first arg in DATE_PART into a Var fixes [#3882](https://github.com/tobymao/sqlglot/pull/3882) *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`2ad9bfe`](https://github.com/tobymao/sqlglot/commit/2ad9bfef71ae707b83f604f16b47aa583d082c3b) - **snowflake**: support table qualification in USING clause *(PR [#3885](https://github.com/tobymao/sqlglot/pull/3885) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3881](https://github.com/tobymao/sqlglot/issues/3881) opened by [@dlahyani](https://github.com/dlahyani)*
- [`ef16b1d`](https://github.com/tobymao/sqlglot/commit/ef16b1da6b43647a0ca08d69eaf3610e3b72671f) - Fix COLLATE's RHS parsing *(PR [#3887](https://github.com/tobymao/sqlglot/pull/3887) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#3880](https://github.com/tobymao/sqlglot/issues/3880) opened by [@ewhitley](https://github.com/ewhitley)*
### :recycle: Refactors
- [`3eb46db`](https://github.com/tobymao/sqlglot/commit/3eb46db5c429f50b5bb6c0c5517a5f7c1084b5ea) - **optimizer**: switch off CSV file schema inference by default *(PR [#3879](https://github.com/tobymao/sqlglot/pull/3879) by [@georgesittas](https://github.com/georgesittas))*
## [v25.9.0] - 2024-08-05
### :boom: BREAKING CHANGES
- due to [`64e187c`](https://github.com/tobymao/sqlglot/commit/64e187c52cd9725ba79e6afbd444382eba9e5827) - transpile postgres impliclitly exploding GENERATE_SERIES proje… *(PR [#3853](https://github.com/tobymao/sqlglot/pull/3853) by [@georgesittas](https://github.com/georgesittas))*:
transpile postgres impliclitly exploding GENERATE_SERIES proje… (#3853)
- due to [`e53e7cc`](https://github.com/tobymao/sqlglot/commit/e53e7cc02a224563d0a61b0a39298d606b9bac80) - Generation of exp.ArrayConcat for 2-arg based dialects *(PR [#3864](https://github.com/tobymao/sqlglot/pull/3864) by [@VaggelisD](https://github.com/VaggelisD))*:
Generation of exp.ArrayConcat for 2-arg based dialects (#3864)
- due to [`659b8bf`](https://github.com/tobymao/sqlglot/commit/659b8bf12e396856d1562ee4678b4f687629e081) - Support for BQ's exp.GenerateDateArray generation *(PR [#3865](https://github.com/tobymao/sqlglot/pull/3865) by [@VaggelisD](https://github.com/VaggelisD))*:
Support for BQ's exp.GenerateDateArray generation (#3865)
### :sparkles: New Features
- [`6afed2a`](https://github.com/tobymao/sqlglot/commit/6afed2aecc0ce186ff6c484b1ad32ac6a2fb61bc) - **duckdb**: Support for exp.TimeDiff generation *(PR [#3856](https://github.com/tobymao/sqlglot/pull/3856) by [@VaggelisD](https://github.com/VaggelisD))*
- [`64e187c`](https://github.com/tobymao/sqlglot/commit/64e187c52cd9725ba79e6afbd444382eba9e5827) - transpile postgres impliclitly exploding GENERATE_SERIES proje… *(PR [#3853](https://github.com/tobymao/sqlglot/pull/3853) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#3818](https://github.com/tobymao/sqlglot/issues/3818) opened by [@wojciechowski-p](https://github.com/wojciechowski-p)*
- [`8a948c8`](https://github.com/tobymao/sqlglot/commit/8a948c805f7534e266557e1aa08bee0982340685) - **teradata**: Parse RENAME TABLE as Command *(PR [#3863](https://github.com/tobymao/sqlglot/pull/3863) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *addresses issue [#3861](https://github.com/tobymao/sqlglot/issues/3861) opened by [@EdouardW](https://github.com/EdouardW)*
- [`659b8bf`](https://github.com/tobymao/sqlglot/commit/659b8bf12e396856d1562ee4678b4f687629e081) - **duckdb**: Support for BQ's exp.GenerateDateArray generation *(PR [#3865](https://github.com/tobymao/sqlglot/pull/3865) by [@VaggelisD](https://github.com/VaggelisD))*
- [`734f54b`](https://github.com/tobymao/sqlglot/commit/734f54bb6ec697a5213f046fbb1e8174b2c31115) - **snowflake**: add support for a a couple of missing clauses in PIVOT clause *(PR [#3867](https://github.com/tobymao/sqlglot/pull/3867) by [@georgesittas](https://github.com/georgesittas))*
### :bug: Bug Fixes
- [`8710763`](https://github.com/tobymao/sqlglot/commit/87107631378b0972115a01cc0bb99dbfc44a66d7) - **presto**: map %W to %A in the TIME_MAPPING *(PR [#3855](https://github.com/tobymao/sqlglot/pull/3855) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#3854](https://github.com/tobymao/sqlglot/issues/3854) opened by [@ddelzell](https://github.com/ddelzell)*
- [`532f3c8`](https://github.com/tobymao/sqlglot/commit/532f3c8714220058170790b13977cc66760841dc) - **duckdb**: Add implicit casts to DATE_DIFF *(PR [#3857](https://github.com/tobymao/sqlglot/pull/3857) by [@VaggelisD](https://github.com/VaggelisD))*
- [`299c4a5`](https://github.com/tobymao/sqlglot/commit/299c4a559dd04047d5a4c4691f8965972842fe7d) - **clickhouse**: Fix SETTINGS parsing *(PR [#3859](https://github.com/tobymao/sqlglot/pull/3859) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#3858](https://github.com/tobymao/sqlglot/issues/3858) opened by [@obazna](https://github.com/obazna)*
- [`810d23d`](https://github.com/tobymao/sqlglot/commit/810d23d4e42f9a7de83015ec425dff9223598219) - **parser**: make assignment parsing more lenient by allowing keyword in LHS *(PR [#3866](https://github.com/tobymao/sqlglot/pull/3866) by [@georgesittas](https://github.com/georgesittas))*
- [`e53e7cc`](https://github.com/tobymao/sqlglot/commit/e53e7cc02a224563d0a61b0a39298d606b9bac80) - Generation of exp.ArrayConcat for 2-arg based dialects *(PR [#3864](https://github.com/tobymao/sqlglot/pull/3864) by [@VaggelisD](https://github.com/VaggelisD))*
- [`813f127`](https://github.com/tobymao/sqlglot/commit/813f127b293e7087d174f3f632b65ba7b24bc9e3) - **duckdb**: Allow DESCRIBE as a _parse_select() path *(PR [#3871](https://github.com/tobymao/sqlglot/pull/3871) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#3869](https://github.com/tobymao/sqlglot/issues/3869) opened by [@cpcloud](https://github.com/cpcloud)*
- [`6ff0c01`](https://github.com/tobymao/sqlglot/commit/6ff0c01a5b8b19e3090b8cf08aabbb4b27425abb) - Fixed size array parsing *(PR [#3870](https://github.com/tobymao/sqlglot/pull/3870) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#3868](https://github.com/tobymao/sqlglot/issues/3868) opened by [@tekumara](https://github.com/tekumara)*
## [v25.8.1] - 2024-07-30
### :bug: Bug Fixes
- [`a295b3a`](https://github.com/tobymao/sqlglot/commit/a295b3adbef0eff0b3f6c3b8b97b1eaa8c13f144) - **tsql**: regression related to CTEs in CREATE VIEW AS statements *(PR [#3852](https://github.com/tobymao/sqlglot/pull/3852) by [@georgesittas](https://github.com/georgesittas))*
## [v25.8.0] - 2024-07-29
### :sparkles: New Features
- [`e37d63a`](https://github.com/tobymao/sqlglot/commit/e37d63a17d4709135c1de7876b2898cf7bd2e641) - **bigquery**: add support for BYTEINT closes [#3838](https://github.com/tobymao/sqlglot/pull/3838) *(commit by [@georgesittas](https://github.com/georgesittas))*
@ -4238,3 +4479,15 @@ Changelog
[v25.7.0]: https://github.com/tobymao/sqlglot/compare/v25.6.1...v25.7.0
[v25.7.1]: https://github.com/tobymao/sqlglot/compare/v25.7.0...v25.7.1
[v25.8.0]: https://github.com/tobymao/sqlglot/compare/v25.7.1...v25.8.0
[v25.8.1]: https://github.com/tobymao/sqlglot/compare/v25.8.0...v25.8.1
[v25.9.0]: https://github.com/tobymao/sqlglot/compare/v25.8.1...v25.9.0
[v25.10.0]: https://github.com/tobymao/sqlglot/compare/v25.9.0...v25.10.0
[v25.11.0]: https://github.com/tobymao/sqlglot/compare/v25.10.0...v25.11.0
[v25.11.1]: https://github.com/tobymao/sqlglot/compare/v25.11.0...v25.11.1
[v25.11.2]: https://github.com/tobymao/sqlglot/compare/v25.11.1...v25.11.2
[v25.11.3]: https://github.com/tobymao/sqlglot/compare/v25.11.2...v25.11.3
[v25.12.0]: https://github.com/tobymao/sqlglot/compare/v25.11.3...v25.12.0
[v25.13.0]: https://github.com/tobymao/sqlglot/compare/v25.12.0...v25.13.0
[v25.14.0]: https://github.com/tobymao/sqlglot/compare/v25.13.0...v25.14.0
[v25.15.0]: https://github.com/tobymao/sqlglot/compare/v25.14.0...v25.15.0
[v25.16.0]: https://github.com/tobymao/sqlglot/compare/v25.15.0...v25.16.0

View file

@ -10,7 +10,7 @@ Syntax [errors](#parser-errors) are highlighted and dialect incompatibilities ca
Learn more about SQLGlot in the API [documentation](https://sqlglot.com/) and the expression tree [primer](https://github.com/tobymao/sqlglot/blob/main/posts/ast_primer.md).
Contributions are very welcome in SQLGlot; read the [contribution guide](https://github.com/tobymao/sqlglot/blob/main/CONTRIBUTING.md) to get started!
Contributions are very welcome in SQLGlot; read the [contribution guide](https://github.com/tobymao/sqlglot/blob/main/CONTRIBUTING.md) and the [onboarding document](https://github.com/tobymao/sqlglot/blob/main/posts/ast_primer.md) to get started!
## Table of Contents

File diff suppressed because one or more lines are too long

View file

@ -76,8 +76,8 @@
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a>
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;25.8.0&#39;</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">25</span><span class="p">,</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;25.16.0&#39;</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">25</span><span class="p">,</span> <span class="mi">16</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
</span></pre></div>
@ -97,7 +97,7 @@
<section id="version">
<div class="attr variable">
<span class="name">version</span><span class="annotation">: str</span> =
<span class="default_value">&#39;25.8.0&#39;</span>
<span class="default_value">&#39;25.16.0&#39;</span>
</div>
@ -109,7 +109,7 @@
<section id="version_tuple">
<div class="attr variable">
<span class="name">version_tuple</span><span class="annotation">: object</span> =
<span class="default_value">(25, 8, 0)</span>
<span class="default_value">(25, 16, 0)</span>
</div>

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View file

@ -48,6 +48,9 @@
<li>
<a class="variable" href="#RisingWave.Generator.SUPPORTS_UESCAPE">SUPPORTS_UESCAPE</a>
</li>
<li>
<a class="variable" href="#RisingWave.Generator.SUPPORTS_NULLABLE_TYPES">SUPPORTS_NULLABLE_TYPES</a>
</li>
<li>
<a class="variable" href="#RisingWave.Generator.AFTER_HAVING_MODIFIER_TRANSFORMS">AFTER_HAVING_MODIFIER_TRANSFORMS</a>
</li>
@ -87,6 +90,9 @@
<li>
<a class="variable" href="#RisingWave.INVERSE_FORMAT_TRIE">INVERSE_FORMAT_TRIE</a>
</li>
<li>
<a class="variable" href="#RisingWave.INVERSE_CREATABLE_KIND_MAPPING">INVERSE_CREATABLE_KIND_MAPPING</a>
</li>
<li>
<a class="variable" href="#RisingWave.ESCAPED_SEQUENCES">ESCAPED_SEQUENCES</a>
</li>
@ -318,6 +324,18 @@
</div>
<div id="RisingWave.INVERSE_CREATABLE_KIND_MAPPING" class="classattr">
<div class="attr variable">
<span class="name">INVERSE_CREATABLE_KIND_MAPPING</span><span class="annotation">: dict[str, str]</span> =
<span class="default_value">{}</span>
</div>
<a class="headerlink" href="#RisingWave.INVERSE_CREATABLE_KIND_MAPPING"></a>
</div>
<div id="RisingWave.ESCAPED_SEQUENCES" class="classattr">
<div class="attr variable">
@ -503,6 +521,9 @@
<dd id="RisingWave.FORCE_EARLY_ALIAS_REF_EXPANSION" class="variable"><a href="dialect.html#Dialect.FORCE_EARLY_ALIAS_REF_EXPANSION">FORCE_EARLY_ALIAS_REF_EXPANSION</a></dd>
<dd id="RisingWave.EXPAND_ALIAS_REFS_EARLY_ONLY_IN_GROUP_BY" class="variable"><a href="dialect.html#Dialect.EXPAND_ALIAS_REFS_EARLY_ONLY_IN_GROUP_BY">EXPAND_ALIAS_REFS_EARLY_ONLY_IN_GROUP_BY</a></dd>
<dd id="RisingWave.SUPPORTS_ORDER_BY_ALL" class="variable"><a href="dialect.html#Dialect.SUPPORTS_ORDER_BY_ALL">SUPPORTS_ORDER_BY_ALL</a></dd>
<dd id="RisingWave.HAS_DISTINCT_ARRAY_CONSTRUCTORS" class="variable"><a href="dialect.html#Dialect.HAS_DISTINCT_ARRAY_CONSTRUCTORS">HAS_DISTINCT_ARRAY_CONSTRUCTORS</a></dd>
<dd id="RisingWave.SUPPORTS_FIXED_SIZE_ARRAYS" class="variable"><a href="dialect.html#Dialect.SUPPORTS_FIXED_SIZE_ARRAYS">SUPPORTS_FIXED_SIZE_ARRAYS</a></dd>
<dd id="RisingWave.CREATABLE_KIND_MAPPING" class="variable"><a href="dialect.html#Dialect.CREATABLE_KIND_MAPPING">CREATABLE_KIND_MAPPING</a></dd>
<dd id="RisingWave.DATE_PART_MAPPING" class="variable"><a href="dialect.html#Dialect.DATE_PART_MAPPING">DATE_PART_MAPPING</a></dd>
<dd id="RisingWave.TYPE_TO_EXPRESSIONS" class="variable"><a href="dialect.html#Dialect.TYPE_TO_EXPRESSIONS">TYPE_TO_EXPRESSIONS</a></dd>
<dd id="RisingWave.ANNOTATORS" class="variable"><a href="dialect.html#Dialect.ANNOTATORS">ANNOTATORS</a></dd>
@ -644,6 +665,18 @@ Default: True</li>
</div>
<div id="RisingWave.Generator.SUPPORTS_NULLABLE_TYPES" class="classattr">
<div class="attr variable">
<span class="name">SUPPORTS_NULLABLE_TYPES</span> =
<span class="default_value">False</span>
</div>
<a class="headerlink" href="#RisingWave.Generator.SUPPORTS_NULLABLE_TYPES"></a>
</div>
<div id="RisingWave.Generator.AFTER_HAVING_MODIFIER_TRANSFORMS" class="classattr">
<div class="attr variable">
@ -712,6 +745,8 @@ Default: True</li>
<dd id="RisingWave.Generator.WITH_PROPERTIES_PREFIX" class="variable"><a href="../generator.html#Generator.WITH_PROPERTIES_PREFIX">WITH_PROPERTIES_PREFIX</a></dd>
<dd id="RisingWave.Generator.QUOTE_JSON_PATH" class="variable"><a href="../generator.html#Generator.QUOTE_JSON_PATH">QUOTE_JSON_PATH</a></dd>
<dd id="RisingWave.Generator.PAD_FILL_PATTERN_IS_REQUIRED" class="variable"><a href="../generator.html#Generator.PAD_FILL_PATTERN_IS_REQUIRED">PAD_FILL_PATTERN_IS_REQUIRED</a></dd>
<dd id="RisingWave.Generator.SUPPORTS_EXPLODING_PROJECTIONS" class="variable"><a href="../generator.html#Generator.SUPPORTS_EXPLODING_PROJECTIONS">SUPPORTS_EXPLODING_PROJECTIONS</a></dd>
<dd id="RisingWave.Generator.SUPPORTS_CONVERT_TIMEZONE" class="variable"><a href="../generator.html#Generator.SUPPORTS_CONVERT_TIMEZONE">SUPPORTS_CONVERT_TIMEZONE</a></dd>
<dd id="RisingWave.Generator.PARSE_JSON_NAME" class="variable"><a href="../generator.html#Generator.PARSE_JSON_NAME">PARSE_JSON_NAME</a></dd>
<dd id="RisingWave.Generator.TIME_PART_SINGULARS" class="variable"><a href="../generator.html#Generator.TIME_PART_SINGULARS">TIME_PART_SINGULARS</a></dd>
<dd id="RisingWave.Generator.TOKEN_MAPPING" class="variable"><a href="../generator.html#Generator.TOKEN_MAPPING">TOKEN_MAPPING</a></dd>
@ -963,7 +998,7 @@ Default: True</li>
<dd id="RisingWave.Generator.altersortkey_sql" class="function"><a href="../generator.html#Generator.altersortkey_sql">altersortkey_sql</a></dd>
<dd id="RisingWave.Generator.renametable_sql" class="function"><a href="../generator.html#Generator.renametable_sql">renametable_sql</a></dd>
<dd id="RisingWave.Generator.renamecolumn_sql" class="function"><a href="../generator.html#Generator.renamecolumn_sql">renamecolumn_sql</a></dd>
<dd id="RisingWave.Generator.altertable_sql" class="function"><a href="../generator.html#Generator.altertable_sql">altertable_sql</a></dd>
<dd id="RisingWave.Generator.alter_sql" class="function"><a href="../generator.html#Generator.alter_sql">alter_sql</a></dd>
<dd id="RisingWave.Generator.add_column_sql" class="function"><a href="../generator.html#Generator.add_column_sql">add_column_sql</a></dd>
<dd id="RisingWave.Generator.droppartition_sql" class="function"><a href="../generator.html#Generator.droppartition_sql">droppartition_sql</a></dd>
<dd id="RisingWave.Generator.addconstraint_sql" class="function"><a href="../generator.html#Generator.addconstraint_sql">addconstraint_sql</a></dd>
@ -1037,7 +1072,6 @@ Default: True</li>
<dd id="RisingWave.Generator.predict_sql" class="function"><a href="../generator.html#Generator.predict_sql">predict_sql</a></dd>
<dd id="RisingWave.Generator.forin_sql" class="function"><a href="../generator.html#Generator.forin_sql">forin_sql</a></dd>
<dd id="RisingWave.Generator.refresh_sql" class="function"><a href="../generator.html#Generator.refresh_sql">refresh_sql</a></dd>
<dd id="RisingWave.Generator.operator_sql" class="function"><a href="../generator.html#Generator.operator_sql">operator_sql</a></dd>
<dd id="RisingWave.Generator.toarray_sql" class="function"><a href="../generator.html#Generator.toarray_sql">toarray_sql</a></dd>
<dd id="RisingWave.Generator.tsordstotime_sql" class="function"><a href="../generator.html#Generator.tsordstotime_sql">tsordstotime_sql</a></dd>
<dd id="RisingWave.Generator.tsordstotimestamp_sql" class="function"><a href="../generator.html#Generator.tsordstotimestamp_sql">tsordstotimestamp_sql</a></dd>
@ -1064,6 +1098,9 @@ Default: True</li>
<dd id="RisingWave.Generator.changes_sql" class="function"><a href="../generator.html#Generator.changes_sql">changes_sql</a></dd>
<dd id="RisingWave.Generator.pad_sql" class="function"><a href="../generator.html#Generator.pad_sql">pad_sql</a></dd>
<dd id="RisingWave.Generator.summarize_sql" class="function"><a href="../generator.html#Generator.summarize_sql">summarize_sql</a></dd>
<dd id="RisingWave.Generator.explodinggenerateseries_sql" class="function"><a href="../generator.html#Generator.explodinggenerateseries_sql">explodinggenerateseries_sql</a></dd>
<dd id="RisingWave.Generator.arrayconcat_sql" class="function"><a href="../generator.html#Generator.arrayconcat_sql">arrayconcat_sql</a></dd>
<dd id="RisingWave.Generator.converttimezone_sql" class="function"><a href="../generator.html#Generator.converttimezone_sql">converttimezone_sql</a></dd>
</div>
<div><dt><a href="postgres.html#Postgres.Generator">sqlglot.dialects.postgres.Postgres.Generator</a></dt>
@ -1083,6 +1120,7 @@ Default: True</li>
<dd id="RisingWave.Generator.MULTI_ARG_DISTINCT" class="variable"><a href="postgres.html#Postgres.Generator.MULTI_ARG_DISTINCT">MULTI_ARG_DISTINCT</a></dd>
<dd id="RisingWave.Generator.CAN_IMPLEMENT_ARRAY_ANY" class="variable"><a href="postgres.html#Postgres.Generator.CAN_IMPLEMENT_ARRAY_ANY">CAN_IMPLEMENT_ARRAY_ANY</a></dd>
<dd id="RisingWave.Generator.COPY_HAS_INTO_KEYWORD" class="variable"><a href="postgres.html#Postgres.Generator.COPY_HAS_INTO_KEYWORD">COPY_HAS_INTO_KEYWORD</a></dd>
<dd id="RisingWave.Generator.ARRAY_CONCAT_IS_VAR_LEN" class="variable"><a href="postgres.html#Postgres.Generator.ARRAY_CONCAT_IS_VAR_LEN">ARRAY_CONCAT_IS_VAR_LEN</a></dd>
<dd id="RisingWave.Generator.SUPPORTED_JSON_PATH_PARTS" class="variable"><a href="postgres.html#Postgres.Generator.SUPPORTED_JSON_PATH_PARTS">SUPPORTED_JSON_PATH_PARTS</a></dd>
<dd id="RisingWave.Generator.TYPE_MAPPING" class="variable"><a href="postgres.html#Postgres.Generator.TYPE_MAPPING">TYPE_MAPPING</a></dd>
<dd id="RisingWave.Generator.TRANSFORMS" class="variable"><a href="postgres.html#Postgres.Generator.TRANSFORMS">TRANSFORMS</a></dd>
@ -1095,6 +1133,7 @@ Default: True</li>
<dd id="RisingWave.Generator.alterset_sql" class="function"><a href="postgres.html#Postgres.Generator.alterset_sql">alterset_sql</a></dd>
<dd id="RisingWave.Generator.datatype_sql" class="function"><a href="postgres.html#Postgres.Generator.datatype_sql">datatype_sql</a></dd>
<dd id="RisingWave.Generator.cast_sql" class="function"><a href="postgres.html#Postgres.Generator.cast_sql">cast_sql</a></dd>
<dd id="RisingWave.Generator.array_sql" class="function"><a href="postgres.html#Postgres.Generator.array_sql">array_sql</a></dd>
</div>
</dl>

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View file

@ -408,21 +408,23 @@
</span><span id="L-74"><a href="#L-74"><span class="linenos">74</span></a> <span class="k">raise</span> <span class="n">ExecuteError</span><span class="p">(</span><span class="s2">&quot;Tables must support the same table args as schema&quot;</span><span class="p">)</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a>
</span><span id="L-76"><a href="#L-76"><span class="linenos">76</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a>
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimization finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimized SQL: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a>
</span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a> <span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">infer_csv_schemas</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a> <span class="p">)</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a>
</span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimization finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimized SQL: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos">83</span></a>
</span><span id="L-84"><a href="#L-84"><span class="linenos">84</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Logical Plan: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos">84</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-85"><a href="#L-85"><span class="linenos">85</span></a>
</span><span id="L-86"><a href="#L-86"><span class="linenos">86</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
</span><span id="L-87"><a href="#L-87"><span class="linenos">87</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos">88</span></a>
</span><span id="L-89"><a href="#L-89"><span class="linenos">89</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Query finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
</span><span id="L-86"><a href="#L-86"><span class="linenos">86</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Logical Plan: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span>
</span><span id="L-87"><a href="#L-87"><span class="linenos">87</span></a>
</span><span id="L-88"><a href="#L-88"><span class="linenos">88</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos">89</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos">90</span></a>
</span><span id="L-91"><a href="#L-91"><span class="linenos">91</span></a> <span class="k">return</span> <span class="n">result</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos">91</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Query finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos">92</span></a>
</span><span id="L-93"><a href="#L-93"><span class="linenos">93</span></a> <span class="k">return</span> <span class="n">result</span>
</span></pre></div>
@ -496,21 +498,23 @@
</span><span id="execute-75"><a href="#execute-75"><span class="linenos">75</span></a> <span class="k">raise</span> <span class="n">ExecuteError</span><span class="p">(</span><span class="s2">&quot;Tables must support the same table args as schema&quot;</span><span class="p">)</span>
</span><span id="execute-76"><a href="#execute-76"><span class="linenos">76</span></a>
</span><span id="execute-77"><a href="#execute-77"><span class="linenos">77</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
</span><span id="execute-78"><a href="#execute-78"><span class="linenos">78</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span><span class="p">)</span>
</span><span id="execute-79"><a href="#execute-79"><span class="linenos">79</span></a>
</span><span id="execute-80"><a href="#execute-80"><span class="linenos">80</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimization finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
</span><span id="execute-81"><a href="#execute-81"><span class="linenos">81</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimized SQL: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
</span><span id="execute-82"><a href="#execute-82"><span class="linenos">82</span></a>
</span><span id="execute-83"><a href="#execute-83"><span class="linenos">83</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="execute-78"><a href="#execute-78"><span class="linenos">78</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">optimize</span><span class="p">(</span>
</span><span id="execute-79"><a href="#execute-79"><span class="linenos">79</span></a> <span class="n">sql</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">infer_csv_schemas</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">read</span>
</span><span id="execute-80"><a href="#execute-80"><span class="linenos">80</span></a> <span class="p">)</span>
</span><span id="execute-81"><a href="#execute-81"><span class="linenos">81</span></a>
</span><span id="execute-82"><a href="#execute-82"><span class="linenos">82</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimization finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
</span><span id="execute-83"><a href="#execute-83"><span class="linenos">83</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Optimized SQL: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span>
</span><span id="execute-84"><a href="#execute-84"><span class="linenos">84</span></a>
</span><span id="execute-85"><a href="#execute-85"><span class="linenos">85</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Logical Plan: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span>
</span><span id="execute-85"><a href="#execute-85"><span class="linenos">85</span></a> <span class="n">plan</span> <span class="o">=</span> <span class="n">Plan</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="execute-86"><a href="#execute-86"><span class="linenos">86</span></a>
</span><span id="execute-87"><a href="#execute-87"><span class="linenos">87</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
</span><span id="execute-88"><a href="#execute-88"><span class="linenos">88</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span>
</span><span id="execute-89"><a href="#execute-89"><span class="linenos">89</span></a>
</span><span id="execute-90"><a href="#execute-90"><span class="linenos">90</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Query finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
</span><span id="execute-87"><a href="#execute-87"><span class="linenos">87</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Logical Plan: </span><span class="si">%s</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">plan</span><span class="p">)</span>
</span><span id="execute-88"><a href="#execute-88"><span class="linenos">88</span></a>
</span><span id="execute-89"><a href="#execute-89"><span class="linenos">89</span></a> <span class="n">now</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
</span><span id="execute-90"><a href="#execute-90"><span class="linenos">90</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">PythonExecutor</span><span class="p">(</span><span class="n">tables</span><span class="o">=</span><span class="n">tables_</span><span class="p">)</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">plan</span><span class="p">)</span>
</span><span id="execute-91"><a href="#execute-91"><span class="linenos">91</span></a>
</span><span id="execute-92"><a href="#execute-92"><span class="linenos">92</span></a> <span class="k">return</span> <span class="n">result</span>
</span><span id="execute-92"><a href="#execute-92"><span class="linenos">92</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">&quot;Query finished: </span><span class="si">%f</span><span class="s2">&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">now</span><span class="p">)</span>
</span><span id="execute-93"><a href="#execute-93"><span class="linenos">93</span></a>
</span><span id="execute-94"><a href="#execute-94"><span class="linenos">94</span></a> <span class="k">return</span> <span class="n">result</span>
</span></pre></div>

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View file

@ -1895,7 +1895,7 @@ belong to some totally-ordered set.</p>
<section id="DATE_UNITS">
<div class="attr variable">
<span class="name">DATE_UNITS</span> =
<span class="default_value">{&#39;day&#39;, &#39;quarter&#39;, &#39;month&#39;, &#39;year_month&#39;, &#39;week&#39;, &#39;year&#39;}</span>
<span class="default_value">{&#39;year&#39;, &#39;quarter&#39;, &#39;week&#39;, &#39;day&#39;, &#39;month&#39;, &#39;year_month&#39;}</span>
</div>

View file

@ -391,6 +391,7 @@
<dd id="JSONPathTokenizer.HEREDOC_TAG_IS_IDENTIFIER" class="variable"><a href="tokens.html#Tokenizer.HEREDOC_TAG_IS_IDENTIFIER">HEREDOC_TAG_IS_IDENTIFIER</a></dd>
<dd id="JSONPathTokenizer.HEREDOC_STRING_ALTERNATIVE" class="variable"><a href="tokens.html#Tokenizer.HEREDOC_STRING_ALTERNATIVE">HEREDOC_STRING_ALTERNATIVE</a></dd>
<dd id="JSONPathTokenizer.STRING_ESCAPES_ALLOWED_IN_RAW_STRINGS" class="variable"><a href="tokens.html#Tokenizer.STRING_ESCAPES_ALLOWED_IN_RAW_STRINGS">STRING_ESCAPES_ALLOWED_IN_RAW_STRINGS</a></dd>
<dd id="JSONPathTokenizer.NESTED_COMMENTS" class="variable"><a href="tokens.html#Tokenizer.NESTED_COMMENTS">NESTED_COMMENTS</a></dd>
<dd id="JSONPathTokenizer.WHITE_SPACE" class="variable"><a href="tokens.html#Tokenizer.WHITE_SPACE">WHITE_SPACE</a></dd>
<dd id="JSONPathTokenizer.COMMANDS" class="variable"><a href="tokens.html#Tokenizer.COMMANDS">COMMANDS</a></dd>
<dd id="JSONPathTokenizer.COMMAND_PREFIX_TOKENS" class="variable"><a href="tokens.html#Tokenizer.COMMAND_PREFIX_TOKENS">COMMAND_PREFIX_TOKENS</a></dd>
@ -585,7 +586,7 @@
<div class="attr variable">
<span class="name">ALL_JSON_PATH_PARTS</span> =
<input id="ALL_JSON_PATH_PARTS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>&#39;&gt;}</span>
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>&#39;&gt;}</span>
</div>

File diff suppressed because one or more lines are too long

View file

@ -119,140 +119,148 @@
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TsOrDsToDate</span><span class="p">))</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;zone&quot;</span><span class="p">)</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> <span class="p">):</span>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;zone&quot;</span><span class="p">):</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.annotate_types</span> <span class="kn">import</span> <span class="n">annotate_types</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">is_string</span>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="ow">and</span> <span class="n">is_iso_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="p">):</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;zone&quot;</span><span class="p">):</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.annotate_types</span> <span class="kn">import</span> <span class="n">annotate_types</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a><span class="n">COERCIBLE_DATE_OPS</span> <span class="o">=</span> <span class="p">(</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Add</span><span class="p">,</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Sub</span><span class="p">,</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">EQ</span><span class="p">,</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NEQ</span><span class="p">,</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GT</span><span class="p">,</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GTE</span><span class="p">,</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LT</span><span class="p">,</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LTE</span><span class="p">,</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeEQ</span><span class="p">,</span>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeNEQ</span><span class="p">,</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a><span class="p">)</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;low&quot;</span><span class="p">])</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="p">):</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a><span class="n">COERCIBLE_DATE_OPS</span> <span class="o">=</span> <span class="p">(</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Add</span><span class="p">,</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Sub</span><span class="p">,</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">EQ</span><span class="p">,</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NEQ</span><span class="p">,</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GT</span><span class="p">,</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">GTE</span><span class="p">,</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LT</span><span class="p">,</span>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">LTE</span><span class="p">,</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeEQ</span><span class="p">,</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">NullSafeNEQ</span><span class="p">,</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a><span class="p">)</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;low&quot;</span><span class="p">])</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="p">):</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="p">):</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="c1"># We can&#39;t replace num in CASE x WHEN num ..., because it&#39;s not the full predicate</span>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="p">):</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="p">):</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TsOrDsToDate</span><span class="p">))</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="p">):</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="c1"># We can&#39;t replace num in CASE x WHEN num ..., because it&#39;s not the full predicate</span>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="p">):</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a><span class="k">def</span> <span class="nf">_coerce_date</span><span class="p">(</span><span class="n">a</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">b</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">for</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">permutations</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">]):</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Interval</span><span class="p">):</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="n">a</span> <span class="o">=</span> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">a</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="ow">and</span> <span class="n">a</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="p">):</span>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a><span class="k">def</span> <span class="nf">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">arg</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">unit</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="k">return</span> <span class="n">arg</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="k">if</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span><span class="p">:</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">date_text</span> <span class="o">=</span> <span class="n">arg</span><span class="o">.</span><span class="n">name</span>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="n">is_iso_date_</span> <span class="o">=</span> <span class="n">is_iso_date</span><span class="p">(</span><span class="n">date_text</span><span class="p">)</span>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a><span class="k">def</span> <span class="nf">_coerce_date</span><span class="p">(</span><span class="n">a</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">b</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">for</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">permutations</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">]):</span>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Interval</span><span class="p">):</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="n">a</span> <span class="o">=</span> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">a</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="ow">and</span> <span class="n">a</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="ow">and</span> <span class="n">b</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="p">):</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">and</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">))</span>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="c1"># An ISO date is also an ISO datetime, but not vice versa</span>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">or</span> <span class="n">is_iso_datetime</span><span class="p">(</span><span class="n">date_text</span><span class="p">):</span>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a>
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="k">elif</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a>
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="k">return</span> <span class="n">arg</span>
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a>
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a>
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a><span class="k">def</span> <span class="nf">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">):</span>
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span><span class="p">:</span>
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="n">e</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a><span class="k">def</span> <span class="nf">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">arg</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">unit</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="k">return</span> <span class="n">arg</span>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="k">if</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEXT_TYPES</span><span class="p">:</span>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">date_text</span> <span class="o">=</span> <span class="n">arg</span><span class="o">.</span><span class="n">name</span>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">is_iso_date_</span> <span class="o">=</span> <span class="n">is_iso_date</span><span class="p">(</span><span class="n">date_text</span><span class="p">)</span>
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a>
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">and</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">))</span>
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a>
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="c1"># An ISO date is also an ISO datetime, but not vice versa</span>
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="k">if</span> <span class="n">is_iso_date_</span> <span class="ow">or</span> <span class="n">is_iso_datetime</span><span class="p">(</span><span class="n">date_text</span><span class="p">):</span>
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a>
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="k">elif</span> <span class="n">arg</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">is_date_unit</span><span class="p">(</span><span class="n">unit</span><span class="p">):</span>
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="k">return</span> <span class="n">arg</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">arg</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a>
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a>
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a><span class="k">def</span> <span class="nf">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">to</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">to</span><span class="p">))</span>
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a>
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a>
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a><span class="c1"># this was originally designed for presto, there is a similar transform for tsql</span>
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a><span class="c1"># this is different in that it only operates on int types, this is because</span>
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a><span class="c1"># presto has a boolean type whereas tsql doesn&#39;t (people use bits)</span>
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a><span class="c1"># with y as (select true as x) select x = 0 FROM y -- illegal presto query</span>
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a><span class="k">def</span> <span class="nf">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Coalesce</span><span class="p">):</span>
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="k">for</span> <span class="n">child</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">iter_expressions</span><span class="p">():</span>
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">child</span><span class="p">)</span>
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="k">elif</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">INTEGER_TYPES</span><span class="p">:</span>
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">neq</span><span class="p">(</span><span class="mi">0</span><span class="p">))</span>
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">return</span> <span class="n">arg</span>
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a>
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a>
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a><span class="k">def</span> <span class="nf">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">):</span>
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="k">if</span> <span class="n">e</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span><span class="p">:</span>
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="n">e</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">))</span>
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a>
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a>
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a><span class="k">def</span> <span class="nf">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">to</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">to</span><span class="o">=</span><span class="n">to</span><span class="p">))</span>
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a>
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a>
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a><span class="c1"># this was originally designed for presto, there is a similar transform for tsql</span>
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a><span class="c1"># this is different in that it only operates on int types, this is because</span>
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a><span class="c1"># presto has a boolean type whereas tsql doesn&#39;t (people use bits)</span>
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a><span class="c1"># with y as (select true as x) select x = 0 FROM y -- illegal presto query</span>
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a><span class="k">def</span> <span class="nf">_replace_int_predicate</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Coalesce</span><span class="p">):</span>
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="k">for</span> <span class="n">child</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">iter_expressions</span><span class="p">():</span>
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="n">_replace_int_predicate</span><span class="p">(</span><span class="n">child</span><span class="p">)</span>
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="k">elif</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">INTEGER_TYPES</span><span class="p">:</span>
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">neq</span><span class="p">(</span><span class="mi">0</span><span class="p">))</span>
</span></pre></div>
@ -342,16 +350,18 @@ conversions rely on type inference.</p>
</span><span id="replace_date_funcs-41"><a href="#replace_date_funcs-41"><span class="linenos">41</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TsOrDsToDate</span><span class="p">))</span>
</span><span id="replace_date_funcs-42"><a href="#replace_date_funcs-42"><span class="linenos">42</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expressions</span>
</span><span id="replace_date_funcs-43"><a href="#replace_date_funcs-43"><span class="linenos">43</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;zone&quot;</span><span class="p">)</span>
</span><span id="replace_date_funcs-44"><a href="#replace_date_funcs-44"><span class="linenos">44</span></a> <span class="p">):</span>
</span><span id="replace_date_funcs-45"><a href="#replace_date_funcs-45"><span class="linenos">45</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
</span><span id="replace_date_funcs-46"><a href="#replace_date_funcs-46"><span class="linenos">46</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;zone&quot;</span><span class="p">):</span>
</span><span id="replace_date_funcs-47"><a href="#replace_date_funcs-47"><span class="linenos">47</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
</span><span id="replace_date_funcs-48"><a href="#replace_date_funcs-48"><span class="linenos">48</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.annotate_types</span> <span class="kn">import</span> <span class="n">annotate_types</span>
</span><span id="replace_date_funcs-49"><a href="#replace_date_funcs-49"><span class="linenos">49</span></a>
</span><span id="replace_date_funcs-50"><a href="#replace_date_funcs-50"><span class="linenos">50</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="replace_date_funcs-51"><a href="#replace_date_funcs-51"><span class="linenos">51</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
</span><span id="replace_date_funcs-52"><a href="#replace_date_funcs-52"><span class="linenos">52</span></a>
</span><span id="replace_date_funcs-53"><a href="#replace_date_funcs-53"><span class="linenos">53</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="replace_date_funcs-44"><a href="#replace_date_funcs-44"><span class="linenos">44</span></a> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">is_string</span>
</span><span id="replace_date_funcs-45"><a href="#replace_date_funcs-45"><span class="linenos">45</span></a> <span class="ow">and</span> <span class="n">is_iso_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
</span><span id="replace_date_funcs-46"><a href="#replace_date_funcs-46"><span class="linenos">46</span></a> <span class="p">):</span>
</span><span id="replace_date_funcs-47"><a href="#replace_date_funcs-47"><span class="linenos">47</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span><span class="p">)</span>
</span><span id="replace_date_funcs-48"><a href="#replace_date_funcs-48"><span class="linenos">48</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Timestamp</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;zone&quot;</span><span class="p">):</span>
</span><span id="replace_date_funcs-49"><a href="#replace_date_funcs-49"><span class="linenos">49</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">type</span><span class="p">:</span>
</span><span id="replace_date_funcs-50"><a href="#replace_date_funcs-50"><span class="linenos">50</span></a> <span class="kn">from</span> <span class="nn">sqlglot.optimizer.annotate_types</span> <span class="kn">import</span> <span class="n">annotate_types</span>
</span><span id="replace_date_funcs-51"><a href="#replace_date_funcs-51"><span class="linenos">51</span></a>
</span><span id="replace_date_funcs-52"><a href="#replace_date_funcs-52"><span class="linenos">52</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">annotate_types</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="replace_date_funcs-53"><a href="#replace_date_funcs-53"><span class="linenos">53</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">to</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">type</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">TIMESTAMP</span><span class="p">)</span>
</span><span id="replace_date_funcs-54"><a href="#replace_date_funcs-54"><span class="linenos">54</span></a>
</span><span id="replace_date_funcs-55"><a href="#replace_date_funcs-55"><span class="linenos">55</span></a> <span class="k">return</span> <span class="n">node</span>
</span></pre></div>
@ -382,21 +392,21 @@ conversions rely on type inference.</p>
</div>
<a class="headerlink" href="#coerce_type"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="coerce_type-70"><a href="#coerce_type-70"><span class="linenos">70</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="coerce_type-71"><a href="#coerce_type-71"><span class="linenos">71</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
</span><span id="coerce_type-72"><a href="#coerce_type-72"><span class="linenos">72</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="coerce_type-73"><a href="#coerce_type-73"><span class="linenos">73</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
</span><span id="coerce_type-74"><a href="#coerce_type-74"><span class="linenos">74</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;low&quot;</span><span class="p">])</span>
</span><span id="coerce_type-75"><a href="#coerce_type-75"><span class="linenos">75</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
</span><span id="coerce_type-76"><a href="#coerce_type-76"><span class="linenos">76</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
</span><span id="coerce_type-77"><a href="#coerce_type-77"><span class="linenos">77</span></a> <span class="p">):</span>
</span><span id="coerce_type-78"><a href="#coerce_type-78"><span class="linenos">78</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
</span><span id="coerce_type-79"><a href="#coerce_type-79"><span class="linenos">79</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
</span><span id="coerce_type-80"><a href="#coerce_type-80"><span class="linenos">80</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="coerce_type-81"><a href="#coerce_type-81"><span class="linenos">81</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
</span><span id="coerce_type-82"><a href="#coerce_type-82"><span class="linenos">82</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="coerce_type-83"><a href="#coerce_type-83"><span class="linenos">83</span></a>
</span><span id="coerce_type-84"><a href="#coerce_type-84"><span class="linenos">84</span></a> <span class="k">return</span> <span class="n">node</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="coerce_type-72"><a href="#coerce_type-72"><span class="linenos">72</span></a><span class="k">def</span> <span class="nf">coerce_type</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="coerce_type-73"><a href="#coerce_type-73"><span class="linenos">73</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">COERCIBLE_DATE_OPS</span><span class="p">):</span>
</span><span id="coerce_type-74"><a href="#coerce_type-74"><span class="linenos">74</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">left</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="coerce_type-75"><a href="#coerce_type-75"><span class="linenos">75</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Between</span><span class="p">):</span>
</span><span id="coerce_type-76"><a href="#coerce_type-76"><span class="linenos">76</span></a> <span class="n">_coerce_date</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">&quot;low&quot;</span><span class="p">])</span>
</span><span id="coerce_type-77"><a href="#coerce_type-77"><span class="linenos">77</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Extract</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">is_type</span><span class="p">(</span>
</span><span id="coerce_type-78"><a href="#coerce_type-78"><span class="linenos">78</span></a> <span class="o">*</span><span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">TEMPORAL_TYPES</span>
</span><span id="coerce_type-79"><a href="#coerce_type-79"><span class="linenos">79</span></a> <span class="p">):</span>
</span><span id="coerce_type-80"><a href="#coerce_type-80"><span class="linenos">80</span></a> <span class="n">_replace_cast</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATETIME</span><span class="p">)</span>
</span><span id="coerce_type-81"><a href="#coerce_type-81"><span class="linenos">81</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">DateAdd</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateSub</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateTrunc</span><span class="p">)):</span>
</span><span id="coerce_type-82"><a href="#coerce_type-82"><span class="linenos">82</span></a> <span class="n">_coerce_timeunit_arg</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">unit</span><span class="p">)</span>
</span><span id="coerce_type-83"><a href="#coerce_type-83"><span class="linenos">83</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DateDiff</span><span class="p">):</span>
</span><span id="coerce_type-84"><a href="#coerce_type-84"><span class="linenos">84</span></a> <span class="n">_coerce_datediff_args</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="coerce_type-85"><a href="#coerce_type-85"><span class="linenos">85</span></a>
</span><span id="coerce_type-86"><a href="#coerce_type-86"><span class="linenos">86</span></a> <span class="k">return</span> <span class="n">node</span>
</span></pre></div>
@ -414,14 +424,20 @@ conversions rely on type inference.</p>
</div>
<a class="headerlink" href="#remove_redundant_casts"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_redundant_casts-87"><a href="#remove_redundant_casts-87"><span class="linenos">87</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="remove_redundant_casts-88"><a href="#remove_redundant_casts-88"><span class="linenos">88</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="remove_redundant_casts-89"><a href="#remove_redundant_casts-89"><span class="linenos">89</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
</span><span id="remove_redundant_casts-90"><a href="#remove_redundant_casts-90"><span class="linenos">90</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
</span><span id="remove_redundant_casts-91"><a href="#remove_redundant_casts-91"><span class="linenos">91</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
</span><span id="remove_redundant_casts-92"><a href="#remove_redundant_casts-92"><span class="linenos">92</span></a> <span class="p">):</span>
</span><span id="remove_redundant_casts-93"><a href="#remove_redundant_casts-93"><span class="linenos">93</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
</span><span id="remove_redundant_casts-94"><a href="#remove_redundant_casts-94"><span class="linenos">94</span></a> <span class="k">return</span> <span class="n">expression</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_redundant_casts-89"><a href="#remove_redundant_casts-89"><span class="linenos"> 89</span></a><span class="k">def</span> <span class="nf">remove_redundant_casts</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="remove_redundant_casts-90"><a href="#remove_redundant_casts-90"><span class="linenos"> 90</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="remove_redundant_casts-91"><a href="#remove_redundant_casts-91"><span class="linenos"> 91</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Cast</span><span class="p">)</span>
</span><span id="remove_redundant_casts-92"><a href="#remove_redundant_casts-92"><span class="linenos"> 92</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
</span><span id="remove_redundant_casts-93"><a href="#remove_redundant_casts-93"><span class="linenos"> 93</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">to</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span>
</span><span id="remove_redundant_casts-94"><a href="#remove_redundant_casts-94"><span class="linenos"> 94</span></a> <span class="p">):</span>
</span><span id="remove_redundant_casts-95"><a href="#remove_redundant_casts-95"><span class="linenos"> 95</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
</span><span id="remove_redundant_casts-96"><a href="#remove_redundant_casts-96"><span class="linenos"> 96</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="remove_redundant_casts-97"><a href="#remove_redundant_casts-97"><span class="linenos"> 97</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Date</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TsOrDsToDate</span><span class="p">))</span>
</span><span id="remove_redundant_casts-98"><a href="#remove_redundant_casts-98"><span class="linenos"> 98</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span>
</span><span id="remove_redundant_casts-99"><a href="#remove_redundant_casts-99"><span class="linenos"> 99</span></a> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">type</span><span class="o">.</span><span class="n">this</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">Type</span><span class="o">.</span><span class="n">DATE</span>
</span><span id="remove_redundant_casts-100"><a href="#remove_redundant_casts-100"><span class="linenos">100</span></a> <span class="p">):</span>
</span><span id="remove_redundant_casts-101"><a href="#remove_redundant_casts-101"><span class="linenos">101</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
</span><span id="remove_redundant_casts-102"><a href="#remove_redundant_casts-102"><span class="linenos">102</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>
@ -439,23 +455,23 @@ conversions rely on type inference.</p>
</div>
<a class="headerlink" href="#ensure_bools"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_bools-97"><a href="#ensure_bools-97"><span class="linenos"> 97</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
</span><span id="ensure_bools-98"><a href="#ensure_bools-98"><span class="linenos"> 98</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
</span><span id="ensure_bools-99"><a href="#ensure_bools-99"><span class="linenos"> 99</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="ensure_bools-100"><a href="#ensure_bools-100"><span class="linenos">100</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
</span><span id="ensure_bools-101"><a href="#ensure_bools-101"><span class="linenos">101</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
</span><span id="ensure_bools-102"><a href="#ensure_bools-102"><span class="linenos">102</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="ensure_bools-103"><a href="#ensure_bools-103"><span class="linenos">103</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
</span><span id="ensure_bools-104"><a href="#ensure_bools-104"><span class="linenos">104</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bools-105"><a href="#ensure_bools-105"><span class="linenos">105</span></a> <span class="c1"># We can&#39;t replace num in CASE x WHEN num ..., because it&#39;s not the full predicate</span>
</span><span id="ensure_bools-106"><a href="#ensure_bools-106"><span class="linenos">106</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
</span><span id="ensure_bools-107"><a href="#ensure_bools-107"><span class="linenos">107</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
</span><span id="ensure_bools-108"><a href="#ensure_bools-108"><span class="linenos">108</span></a> <span class="p">):</span>
</span><span id="ensure_bools-109"><a href="#ensure_bools-109"><span class="linenos">109</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bools-110"><a href="#ensure_bools-110"><span class="linenos">110</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
</span><span id="ensure_bools-111"><a href="#ensure_bools-111"><span class="linenos">111</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bools-112"><a href="#ensure_bools-112"><span class="linenos">112</span></a>
</span><span id="ensure_bools-113"><a href="#ensure_bools-113"><span class="linenos">113</span></a> <span class="k">return</span> <span class="n">expression</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_bools-105"><a href="#ensure_bools-105"><span class="linenos">105</span></a><span class="k">def</span> <span class="nf">ensure_bools</span><span class="p">(</span>
</span><span id="ensure_bools-106"><a href="#ensure_bools-106"><span class="linenos">106</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">replace_func</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Callable</span><span class="p">[[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="kc">None</span><span class="p">]</span>
</span><span id="ensure_bools-107"><a href="#ensure_bools-107"><span class="linenos">107</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="ensure_bools-108"><a href="#ensure_bools-108"><span class="linenos">108</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
</span><span id="ensure_bools-109"><a href="#ensure_bools-109"><span class="linenos">109</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
</span><span id="ensure_bools-110"><a href="#ensure_bools-110"><span class="linenos">110</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
</span><span id="ensure_bools-111"><a href="#ensure_bools-111"><span class="linenos">111</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">):</span>
</span><span id="ensure_bools-112"><a href="#ensure_bools-112"><span class="linenos">112</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bools-113"><a href="#ensure_bools-113"><span class="linenos">113</span></a> <span class="c1"># We can&#39;t replace num in CASE x WHEN num ..., because it&#39;s not the full predicate</span>
</span><span id="ensure_bools-114"><a href="#ensure_bools-114"><span class="linenos">114</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">If</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
</span><span id="ensure_bools-115"><a href="#ensure_bools-115"><span class="linenos">115</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Case</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">this</span>
</span><span id="ensure_bools-116"><a href="#ensure_bools-116"><span class="linenos">116</span></a> <span class="p">):</span>
</span><span id="ensure_bools-117"><a href="#ensure_bools-117"><span class="linenos">117</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bools-118"><a href="#ensure_bools-118"><span class="linenos">118</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">)):</span>
</span><span id="ensure_bools-119"><a href="#ensure_bools-119"><span class="linenos">119</span></a> <span class="n">replace_func</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
</span><span id="ensure_bools-120"><a href="#ensure_bools-120"><span class="linenos">120</span></a>
</span><span id="ensure_bools-121"><a href="#ensure_bools-121"><span class="linenos">121</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>
@ -473,12 +489,12 @@ conversions rely on type inference.</p>
</div>
<a class="headerlink" href="#remove_ascending_order"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_ascending_order-116"><a href="#remove_ascending_order-116"><span class="linenos">116</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="remove_ascending_order-117"><a href="#remove_ascending_order-117"><span class="linenos">117</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
</span><span id="remove_ascending_order-118"><a href="#remove_ascending_order-118"><span class="linenos">118</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
</span><span id="remove_ascending_order-119"><a href="#remove_ascending_order-119"><span class="linenos">119</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="remove_ascending_order-120"><a href="#remove_ascending_order-120"><span class="linenos">120</span></a>
</span><span id="remove_ascending_order-121"><a href="#remove_ascending_order-121"><span class="linenos">121</span></a> <span class="k">return</span> <span class="n">expression</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="remove_ascending_order-124"><a href="#remove_ascending_order-124"><span class="linenos">124</span></a><span class="k">def</span> <span class="nf">remove_ascending_order</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="remove_ascending_order-125"><a href="#remove_ascending_order-125"><span class="linenos">125</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Ordered</span><span class="p">)</span> <span class="ow">and</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">)</span> <span class="ow">is</span> <span class="kc">False</span><span class="p">:</span>
</span><span id="remove_ascending_order-126"><a href="#remove_ascending_order-126"><span class="linenos">126</span></a> <span class="c1"># Convert ORDER BY a ASC to ORDER BY a</span>
</span><span id="remove_ascending_order-127"><a href="#remove_ascending_order-127"><span class="linenos">127</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;desc&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="remove_ascending_order-128"><a href="#remove_ascending_order-128"><span class="linenos">128</span></a>
</span><span id="remove_ascending_order-129"><a href="#remove_ascending_order-129"><span class="linenos">129</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>

View file

@ -586,7 +586,7 @@ queries if it would result in multiple table selects in a single query:</p>
<div class="attr variable">
<span class="name">UNMERGABLE_ARGS</span> =
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;limit&#39;, &#39;with&#39;, &#39;into&#39;, &#39;match&#39;, &#39;pivots&#39;, &#39;group&#39;, &#39;locks&#39;, &#39;prewhere&#39;, &#39;kind&#39;, &#39;connect&#39;, &#39;having&#39;, &#39;settings&#39;, &#39;windows&#39;, &#39;options&#39;, &#39;sort&#39;, &#39;laterals&#39;, &#39;qualify&#39;, &#39;format&#39;, &#39;cluster&#39;, &#39;distinct&#39;, &#39;offset&#39;, &#39;sample&#39;, &#39;distribute&#39;}</span>
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;connect&#39;, &#39;options&#39;, &#39;group&#39;, &#39;having&#39;, &#39;qualify&#39;, &#39;sample&#39;, &#39;offset&#39;, &#39;with&#39;, &#39;cluster&#39;, &#39;limit&#39;, &#39;settings&#39;, &#39;distinct&#39;, &#39;sort&#39;, &#39;match&#39;, &#39;windows&#39;, &#39;format&#39;, &#39;pivots&#39;, &#39;kind&#39;, &#39;prewhere&#39;, &#39;into&#39;, &#39;laterals&#39;, &#39;locks&#39;, &#39;distribute&#39;}</span>
</div>

View file

@ -77,45 +77,49 @@
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a>
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a><span class="k">def</span> <span class="nf">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-22"><a href="#L-22"><span class="linenos">22</span></a><span class="sd"> Normalize all unquoted identifiers to either lower or upper case, depending</span>
</span><span id="L-23"><a href="#L-23"><span class="linenos">23</span></a><span class="sd"> on the dialect. This essentially makes those identifiers case-insensitive.</span>
</span><span id="L-24"><a href="#L-24"><span class="linenos">24</span></a>
</span><span id="L-25"><a href="#L-25"><span class="linenos">25</span></a><span class="sd"> It&#39;s possible to make this a no-op by adding a special comment next to the</span>
</span><span id="L-26"><a href="#L-26"><span class="linenos">26</span></a><span class="sd"> identifier of interest:</span>
</span><span id="L-27"><a href="#L-27"><span class="linenos">27</span></a>
</span><span id="L-28"><a href="#L-28"><span class="linenos">28</span></a><span class="sd"> SELECT a /* sqlglot.meta case_sensitive */ FROM table</span>
</span><span id="L-29"><a href="#L-29"><span class="linenos">29</span></a>
</span><span id="L-30"><a href="#L-30"><span class="linenos">30</span></a><span class="sd"> In this example, the identifier `a` will not be normalized.</span>
</span><span id="L-22"><a href="#L-22"><span class="linenos">22</span></a><span class="sd"> Normalize identifiers by converting them to either lower or upper case,</span>
</span><span id="L-23"><a href="#L-23"><span class="linenos">23</span></a><span class="sd"> ensuring the semantics are preserved in each case (e.g. by respecting</span>
</span><span id="L-24"><a href="#L-24"><span class="linenos">24</span></a><span class="sd"> case-sensitivity).</span>
</span><span id="L-25"><a href="#L-25"><span class="linenos">25</span></a>
</span><span id="L-26"><a href="#L-26"><span class="linenos">26</span></a><span class="sd"> This transformation reflects how identifiers would be resolved by the engine corresponding</span>
</span><span id="L-27"><a href="#L-27"><span class="linenos">27</span></a><span class="sd"> to each SQL dialect, and plays a very important role in the standardization of the AST.</span>
</span><span id="L-28"><a href="#L-28"><span class="linenos">28</span></a>
</span><span id="L-29"><a href="#L-29"><span class="linenos">29</span></a><span class="sd"> It&#39;s possible to make this a no-op by adding a special comment next to the</span>
</span><span id="L-30"><a href="#L-30"><span class="linenos">30</span></a><span class="sd"> identifier of interest:</span>
</span><span id="L-31"><a href="#L-31"><span class="linenos">31</span></a>
</span><span id="L-32"><a href="#L-32"><span class="linenos">32</span></a><span class="sd"> Note:</span>
</span><span id="L-33"><a href="#L-33"><span class="linenos">33</span></a><span class="sd"> Some dialects (e.g. BigQuery) treat identifiers as case-insensitive even</span>
</span><span id="L-34"><a href="#L-34"><span class="linenos">34</span></a><span class="sd"> when they&#39;re quoted, so in these cases all identifiers are normalized.</span>
</span><span id="L-32"><a href="#L-32"><span class="linenos">32</span></a><span class="sd"> SELECT a /* sqlglot.meta case_sensitive */ FROM table</span>
</span><span id="L-33"><a href="#L-33"><span class="linenos">33</span></a>
</span><span id="L-34"><a href="#L-34"><span class="linenos">34</span></a><span class="sd"> In this example, the identifier `a` will not be normalized.</span>
</span><span id="L-35"><a href="#L-35"><span class="linenos">35</span></a>
</span><span id="L-36"><a href="#L-36"><span class="linenos">36</span></a><span class="sd"> Example:</span>
</span><span id="L-37"><a href="#L-37"><span class="linenos">37</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="L-38"><a href="#L-38"><span class="linenos">38</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&#39;SELECT Bar.A AS A FROM &quot;Foo&quot;.Bar&#39;)</span>
</span><span id="L-39"><a href="#L-39"><span class="linenos">39</span></a><span class="sd"> &gt;&gt;&gt; normalize_identifiers(expression).sql()</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos">40</span></a><span class="sd"> &#39;SELECT bar.a AS a FROM &quot;Foo&quot;.bar&#39;</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos">41</span></a><span class="sd"> &gt;&gt;&gt; normalize_identifiers(&quot;foo&quot;, dialect=&quot;snowflake&quot;).sql(dialect=&quot;snowflake&quot;)</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos">42</span></a><span class="sd"> &#39;FOO&#39;</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos">43</span></a>
</span><span id="L-44"><a href="#L-44"><span class="linenos">44</span></a><span class="sd"> Args:</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos">45</span></a><span class="sd"> expression: The expression to transform.</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos">46</span></a><span class="sd"> dialect: The dialect to use in order to decide how to normalize identifiers.</span>
</span><span id="L-36"><a href="#L-36"><span class="linenos">36</span></a><span class="sd"> Note:</span>
</span><span id="L-37"><a href="#L-37"><span class="linenos">37</span></a><span class="sd"> Some dialects (e.g. DuckDB) treat all identifiers as case-insensitive even</span>
</span><span id="L-38"><a href="#L-38"><span class="linenos">38</span></a><span class="sd"> when they&#39;re quoted, so in these cases all identifiers are normalized.</span>
</span><span id="L-39"><a href="#L-39"><span class="linenos">39</span></a>
</span><span id="L-40"><a href="#L-40"><span class="linenos">40</span></a><span class="sd"> Example:</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos">41</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos">42</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&#39;SELECT Bar.A AS A FROM &quot;Foo&quot;.Bar&#39;)</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos">43</span></a><span class="sd"> &gt;&gt;&gt; normalize_identifiers(expression).sql()</span>
</span><span id="L-44"><a href="#L-44"><span class="linenos">44</span></a><span class="sd"> &#39;SELECT bar.a AS a FROM &quot;Foo&quot;.bar&#39;</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos">45</span></a><span class="sd"> &gt;&gt;&gt; normalize_identifiers(&quot;foo&quot;, dialect=&quot;snowflake&quot;).sql(dialect=&quot;snowflake&quot;)</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos">46</span></a><span class="sd"> &#39;FOO&#39;</span>
</span><span id="L-47"><a href="#L-47"><span class="linenos">47</span></a>
</span><span id="L-48"><a href="#L-48"><span class="linenos">48</span></a><span class="sd"> Returns:</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a><span class="sd"> The transformed expression.</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a>
</span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a>
</span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">n</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">)):</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">):</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos">48</span></a><span class="sd"> Args:</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a><span class="sd"> expression: The expression to transform.</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a><span class="sd"> dialect: The dialect to use in order to decide how to normalize identifiers.</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a>
</span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a><span class="sd"> Returns:</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a><span class="sd"> The transformed expression.</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a>
</span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a>
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">n</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">)):</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">):</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos">63</span></a>
</span><span id="L-64"><a href="#L-64"><span class="linenos">64</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>
@ -133,50 +137,58 @@
<a class="headerlink" href="#normalize_identifiers"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="normalize_identifiers-21"><a href="#normalize_identifiers-21"><span class="linenos">21</span></a><span class="k">def</span> <span class="nf">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="normalize_identifiers-22"><a href="#normalize_identifiers-22"><span class="linenos">22</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="normalize_identifiers-23"><a href="#normalize_identifiers-23"><span class="linenos">23</span></a><span class="sd"> Normalize all unquoted identifiers to either lower or upper case, depending</span>
</span><span id="normalize_identifiers-24"><a href="#normalize_identifiers-24"><span class="linenos">24</span></a><span class="sd"> on the dialect. This essentially makes those identifiers case-insensitive.</span>
</span><span id="normalize_identifiers-25"><a href="#normalize_identifiers-25"><span class="linenos">25</span></a>
</span><span id="normalize_identifiers-26"><a href="#normalize_identifiers-26"><span class="linenos">26</span></a><span class="sd"> It&#39;s possible to make this a no-op by adding a special comment next to the</span>
</span><span id="normalize_identifiers-27"><a href="#normalize_identifiers-27"><span class="linenos">27</span></a><span class="sd"> identifier of interest:</span>
</span><span id="normalize_identifiers-28"><a href="#normalize_identifiers-28"><span class="linenos">28</span></a>
</span><span id="normalize_identifiers-29"><a href="#normalize_identifiers-29"><span class="linenos">29</span></a><span class="sd"> SELECT a /* sqlglot.meta case_sensitive */ FROM table</span>
</span><span id="normalize_identifiers-30"><a href="#normalize_identifiers-30"><span class="linenos">30</span></a>
</span><span id="normalize_identifiers-31"><a href="#normalize_identifiers-31"><span class="linenos">31</span></a><span class="sd"> In this example, the identifier `a` will not be normalized.</span>
</span><span id="normalize_identifiers-23"><a href="#normalize_identifiers-23"><span class="linenos">23</span></a><span class="sd"> Normalize identifiers by converting them to either lower or upper case,</span>
</span><span id="normalize_identifiers-24"><a href="#normalize_identifiers-24"><span class="linenos">24</span></a><span class="sd"> ensuring the semantics are preserved in each case (e.g. by respecting</span>
</span><span id="normalize_identifiers-25"><a href="#normalize_identifiers-25"><span class="linenos">25</span></a><span class="sd"> case-sensitivity).</span>
</span><span id="normalize_identifiers-26"><a href="#normalize_identifiers-26"><span class="linenos">26</span></a>
</span><span id="normalize_identifiers-27"><a href="#normalize_identifiers-27"><span class="linenos">27</span></a><span class="sd"> This transformation reflects how identifiers would be resolved by the engine corresponding</span>
</span><span id="normalize_identifiers-28"><a href="#normalize_identifiers-28"><span class="linenos">28</span></a><span class="sd"> to each SQL dialect, and plays a very important role in the standardization of the AST.</span>
</span><span id="normalize_identifiers-29"><a href="#normalize_identifiers-29"><span class="linenos">29</span></a>
</span><span id="normalize_identifiers-30"><a href="#normalize_identifiers-30"><span class="linenos">30</span></a><span class="sd"> It&#39;s possible to make this a no-op by adding a special comment next to the</span>
</span><span id="normalize_identifiers-31"><a href="#normalize_identifiers-31"><span class="linenos">31</span></a><span class="sd"> identifier of interest:</span>
</span><span id="normalize_identifiers-32"><a href="#normalize_identifiers-32"><span class="linenos">32</span></a>
</span><span id="normalize_identifiers-33"><a href="#normalize_identifiers-33"><span class="linenos">33</span></a><span class="sd"> Note:</span>
</span><span id="normalize_identifiers-34"><a href="#normalize_identifiers-34"><span class="linenos">34</span></a><span class="sd"> Some dialects (e.g. BigQuery) treat identifiers as case-insensitive even</span>
</span><span id="normalize_identifiers-35"><a href="#normalize_identifiers-35"><span class="linenos">35</span></a><span class="sd"> when they&#39;re quoted, so in these cases all identifiers are normalized.</span>
</span><span id="normalize_identifiers-33"><a href="#normalize_identifiers-33"><span class="linenos">33</span></a><span class="sd"> SELECT a /* sqlglot.meta case_sensitive */ FROM table</span>
</span><span id="normalize_identifiers-34"><a href="#normalize_identifiers-34"><span class="linenos">34</span></a>
</span><span id="normalize_identifiers-35"><a href="#normalize_identifiers-35"><span class="linenos">35</span></a><span class="sd"> In this example, the identifier `a` will not be normalized.</span>
</span><span id="normalize_identifiers-36"><a href="#normalize_identifiers-36"><span class="linenos">36</span></a>
</span><span id="normalize_identifiers-37"><a href="#normalize_identifiers-37"><span class="linenos">37</span></a><span class="sd"> Example:</span>
</span><span id="normalize_identifiers-38"><a href="#normalize_identifiers-38"><span class="linenos">38</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="normalize_identifiers-39"><a href="#normalize_identifiers-39"><span class="linenos">39</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&#39;SELECT Bar.A AS A FROM &quot;Foo&quot;.Bar&#39;)</span>
</span><span id="normalize_identifiers-40"><a href="#normalize_identifiers-40"><span class="linenos">40</span></a><span class="sd"> &gt;&gt;&gt; normalize_identifiers(expression).sql()</span>
</span><span id="normalize_identifiers-41"><a href="#normalize_identifiers-41"><span class="linenos">41</span></a><span class="sd"> &#39;SELECT bar.a AS a FROM &quot;Foo&quot;.bar&#39;</span>
</span><span id="normalize_identifiers-42"><a href="#normalize_identifiers-42"><span class="linenos">42</span></a><span class="sd"> &gt;&gt;&gt; normalize_identifiers(&quot;foo&quot;, dialect=&quot;snowflake&quot;).sql(dialect=&quot;snowflake&quot;)</span>
</span><span id="normalize_identifiers-43"><a href="#normalize_identifiers-43"><span class="linenos">43</span></a><span class="sd"> &#39;FOO&#39;</span>
</span><span id="normalize_identifiers-44"><a href="#normalize_identifiers-44"><span class="linenos">44</span></a>
</span><span id="normalize_identifiers-45"><a href="#normalize_identifiers-45"><span class="linenos">45</span></a><span class="sd"> Args:</span>
</span><span id="normalize_identifiers-46"><a href="#normalize_identifiers-46"><span class="linenos">46</span></a><span class="sd"> expression: The expression to transform.</span>
</span><span id="normalize_identifiers-47"><a href="#normalize_identifiers-47"><span class="linenos">47</span></a><span class="sd"> dialect: The dialect to use in order to decide how to normalize identifiers.</span>
</span><span id="normalize_identifiers-37"><a href="#normalize_identifiers-37"><span class="linenos">37</span></a><span class="sd"> Note:</span>
</span><span id="normalize_identifiers-38"><a href="#normalize_identifiers-38"><span class="linenos">38</span></a><span class="sd"> Some dialects (e.g. DuckDB) treat all identifiers as case-insensitive even</span>
</span><span id="normalize_identifiers-39"><a href="#normalize_identifiers-39"><span class="linenos">39</span></a><span class="sd"> when they&#39;re quoted, so in these cases all identifiers are normalized.</span>
</span><span id="normalize_identifiers-40"><a href="#normalize_identifiers-40"><span class="linenos">40</span></a>
</span><span id="normalize_identifiers-41"><a href="#normalize_identifiers-41"><span class="linenos">41</span></a><span class="sd"> Example:</span>
</span><span id="normalize_identifiers-42"><a href="#normalize_identifiers-42"><span class="linenos">42</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="normalize_identifiers-43"><a href="#normalize_identifiers-43"><span class="linenos">43</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&#39;SELECT Bar.A AS A FROM &quot;Foo&quot;.Bar&#39;)</span>
</span><span id="normalize_identifiers-44"><a href="#normalize_identifiers-44"><span class="linenos">44</span></a><span class="sd"> &gt;&gt;&gt; normalize_identifiers(expression).sql()</span>
</span><span id="normalize_identifiers-45"><a href="#normalize_identifiers-45"><span class="linenos">45</span></a><span class="sd"> &#39;SELECT bar.a AS a FROM &quot;Foo&quot;.bar&#39;</span>
</span><span id="normalize_identifiers-46"><a href="#normalize_identifiers-46"><span class="linenos">46</span></a><span class="sd"> &gt;&gt;&gt; normalize_identifiers(&quot;foo&quot;, dialect=&quot;snowflake&quot;).sql(dialect=&quot;snowflake&quot;)</span>
</span><span id="normalize_identifiers-47"><a href="#normalize_identifiers-47"><span class="linenos">47</span></a><span class="sd"> &#39;FOO&#39;</span>
</span><span id="normalize_identifiers-48"><a href="#normalize_identifiers-48"><span class="linenos">48</span></a>
</span><span id="normalize_identifiers-49"><a href="#normalize_identifiers-49"><span class="linenos">49</span></a><span class="sd"> Returns:</span>
</span><span id="normalize_identifiers-50"><a href="#normalize_identifiers-50"><span class="linenos">50</span></a><span class="sd"> The transformed expression.</span>
</span><span id="normalize_identifiers-51"><a href="#normalize_identifiers-51"><span class="linenos">51</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="normalize_identifiers-52"><a href="#normalize_identifiers-52"><span class="linenos">52</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="normalize_identifiers-53"><a href="#normalize_identifiers-53"><span class="linenos">53</span></a>
</span><span id="normalize_identifiers-54"><a href="#normalize_identifiers-54"><span class="linenos">54</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
</span><span id="normalize_identifiers-55"><a href="#normalize_identifiers-55"><span class="linenos">55</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="normalize_identifiers-56"><a href="#normalize_identifiers-56"><span class="linenos">56</span></a>
</span><span id="normalize_identifiers-57"><a href="#normalize_identifiers-57"><span class="linenos">57</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">n</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">)):</span>
</span><span id="normalize_identifiers-58"><a href="#normalize_identifiers-58"><span class="linenos">58</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">):</span>
</span><span id="normalize_identifiers-59"><a href="#normalize_identifiers-59"><span class="linenos">59</span></a> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="normalize_identifiers-49"><a href="#normalize_identifiers-49"><span class="linenos">49</span></a><span class="sd"> Args:</span>
</span><span id="normalize_identifiers-50"><a href="#normalize_identifiers-50"><span class="linenos">50</span></a><span class="sd"> expression: The expression to transform.</span>
</span><span id="normalize_identifiers-51"><a href="#normalize_identifiers-51"><span class="linenos">51</span></a><span class="sd"> dialect: The dialect to use in order to decide how to normalize identifiers.</span>
</span><span id="normalize_identifiers-52"><a href="#normalize_identifiers-52"><span class="linenos">52</span></a>
</span><span id="normalize_identifiers-53"><a href="#normalize_identifiers-53"><span class="linenos">53</span></a><span class="sd"> Returns:</span>
</span><span id="normalize_identifiers-54"><a href="#normalize_identifiers-54"><span class="linenos">54</span></a><span class="sd"> The transformed expression.</span>
</span><span id="normalize_identifiers-55"><a href="#normalize_identifiers-55"><span class="linenos">55</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="normalize_identifiers-56"><a href="#normalize_identifiers-56"><span class="linenos">56</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="normalize_identifiers-57"><a href="#normalize_identifiers-57"><span class="linenos">57</span></a>
</span><span id="normalize_identifiers-58"><a href="#normalize_identifiers-58"><span class="linenos">58</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
</span><span id="normalize_identifiers-59"><a href="#normalize_identifiers-59"><span class="linenos">59</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="normalize_identifiers-60"><a href="#normalize_identifiers-60"><span class="linenos">60</span></a>
</span><span id="normalize_identifiers-61"><a href="#normalize_identifiers-61"><span class="linenos">61</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="normalize_identifiers-61"><a href="#normalize_identifiers-61"><span class="linenos">61</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">n</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">)):</span>
</span><span id="normalize_identifiers-62"><a href="#normalize_identifiers-62"><span class="linenos">62</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">):</span>
</span><span id="normalize_identifiers-63"><a href="#normalize_identifiers-63"><span class="linenos">63</span></a> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="normalize_identifiers-64"><a href="#normalize_identifiers-64"><span class="linenos">64</span></a>
</span><span id="normalize_identifiers-65"><a href="#normalize_identifiers-65"><span class="linenos">65</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>
<div class="docstring"><p>Normalize all unquoted identifiers to either lower or upper case, depending
on the dialect. This essentially makes those identifiers case-insensitive.</p>
<div class="docstring"><p>Normalize identifiers by converting them to either lower or upper case,
ensuring the semantics are preserved in each case (e.g. by respecting
case-sensitivity).</p>
<p>This transformation reflects how identifiers would be resolved by the engine corresponding
to each SQL dialect, and plays a very important role in the standardization of the AST.</p>
<p>It's possible to make this a no-op by adding a special comment next to the
identifier of interest:</p>
@ -189,7 +201,7 @@ identifier of interest:</p>
<h6 id="note">Note:</h6>
<blockquote>
<p>Some dialects (e.g. BigQuery) treat identifiers as case-insensitive even
<p>Some dialects (e.g. DuckDB) treat all identifiers as case-insensitive even
when they're quoted, so in these cases all identifiers are normalized.</p>
</blockquote>

View file

@ -65,89 +65,98 @@
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.isolate_table_selects</span> <span class="kn">import</span> <span class="n">isolate_table_selects</span>
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.normalize_identifiers</span> <span class="kn">import</span> <span class="n">normalize_identifiers</span>
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.qualify_columns</span> <span class="kn">import</span> <span class="p">(</span>
</span><span id="L-10"><a href="#L-10"><span class="linenos">10</span></a> <span class="n">pushdown_cte_alias_columns</span> <span class="k">as</span> <span class="n">pushdown_cte_alias_columns_func</span><span class="p">,</span>
</span><span id="L-11"><a href="#L-11"><span class="linenos">11</span></a> <span class="n">qualify_columns</span> <span class="k">as</span> <span class="n">qualify_columns_func</span><span class="p">,</span>
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a> <span class="n">quote_identifiers</span> <span class="k">as</span> <span class="n">quote_identifiers_func</span><span class="p">,</span>
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a> <span class="n">validate_qualify_columns</span> <span class="k">as</span> <span class="n">validate_qualify_columns_func</span><span class="p">,</span>
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a><span class="p">)</span>
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.qualify_tables</span> <span class="kn">import</span> <span class="n">qualify_tables</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="kn">from</span> <span class="nn">sqlglot.schema</span> <span class="kn">import</span> <span class="n">Schema</span><span class="p">,</span> <span class="n">ensure_schema</span>
</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a>
</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a>
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a><span class="k">def</span> <span class="nf">qualify</span><span class="p">(</span>
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-22"><a href="#L-22"><span class="linenos">22</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-23"><a href="#L-23"><span class="linenos">23</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-24"><a href="#L-24"><span class="linenos">24</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">dict</span> <span class="o">|</span> <span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-25"><a href="#L-25"><span class="linenos">25</span></a> <span class="n">expand_alias_refs</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="L-26"><a href="#L-26"><span class="linenos">26</span></a> <span class="n">expand_stars</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="L-27"><a href="#L-27"><span class="linenos">27</span></a> <span class="n">infer_schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-28"><a href="#L-28"><span class="linenos">28</span></a> <span class="n">isolate_tables</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="L-29"><a href="#L-29"><span class="linenos">29</span></a> <span class="n">qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="L-30"><a href="#L-30"><span class="linenos">30</span></a> <span class="n">validate_qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="L-31"><a href="#L-31"><span class="linenos">31</span></a> <span class="n">quote_identifiers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="L-32"><a href="#L-32"><span class="linenos">32</span></a> <span class="n">identify</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="L-33"><a href="#L-33"><span class="linenos">33</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-34"><a href="#L-34"><span class="linenos">34</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-35"><a href="#L-35"><span class="linenos">35</span></a><span class="sd"> Rewrite sqlglot AST to have normalized and qualified tables and columns.</span>
</span><span id="L-36"><a href="#L-36"><span class="linenos">36</span></a>
</span><span id="L-37"><a href="#L-37"><span class="linenos">37</span></a><span class="sd"> This step is necessary for all further SQLGlot optimizations.</span>
</span><span id="L-38"><a href="#L-38"><span class="linenos">38</span></a>
</span><span id="L-39"><a href="#L-39"><span class="linenos">39</span></a><span class="sd"> Example:</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos">40</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos">41</span></a><span class="sd"> &gt;&gt;&gt; schema = {&quot;tbl&quot;: {&quot;col&quot;: &quot;INT&quot;}}</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos">42</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT col FROM tbl&quot;)</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos">43</span></a><span class="sd"> &gt;&gt;&gt; qualify(expression, schema=schema).sql()</span>
</span><span id="L-44"><a href="#L-44"><span class="linenos">44</span></a><span class="sd"> &#39;SELECT &quot;tbl&quot;.&quot;col&quot; AS &quot;col&quot; FROM &quot;tbl&quot; AS &quot;tbl&quot;&#39;</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos">45</span></a>
</span><span id="L-46"><a href="#L-46"><span class="linenos">46</span></a><span class="sd"> Args:</span>
</span><span id="L-47"><a href="#L-47"><span class="linenos">47</span></a><span class="sd"> expression: Expression to qualify.</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos">48</span></a><span class="sd"> db: Default database name for tables.</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a><span class="sd"> catalog: Default catalog name for tables.</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a><span class="sd"> schema: Schema to infer column names and types.</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a><span class="sd"> expand_alias_refs: Whether to expand references to aliases.</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a><span class="sd"> expand_stars: Whether to expand star queries. This is a necessary step</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a><span class="sd"> for most of the optimizer&#39;s rules to work; do not set to False unless you</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a><span class="sd"> know what you&#39;re doing!</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a><span class="sd"> infer_schema: Whether to infer the schema if missing.</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a><span class="sd"> isolate_tables: Whether to isolate table selects.</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a><span class="sd"> qualify_columns: Whether to qualify columns.</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a><span class="sd"> validate_qualify_columns: Whether to validate columns.</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a><span class="sd"> quote_identifiers: Whether to run the quote_identifiers step.</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a><span class="sd"> This step is necessary to ensure correctness for case sensitive queries.</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a><span class="sd"> But this flag is provided in case this step is performed at a later time.</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a><span class="sd"> identify: If True, quote all identifiers, else only necessary ones.</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos">63</span></a>
</span><span id="L-64"><a href="#L-64"><span class="linenos">64</span></a><span class="sd"> Returns:</span>
</span><span id="L-65"><a href="#L-65"><span class="linenos">65</span></a><span class="sd"> The qualified expression.</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos">66</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos">67</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos">68</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos">69</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_tables</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">db</span><span class="o">=</span><span class="n">db</span><span class="p">,</span> <span class="n">catalog</span><span class="o">=</span><span class="n">catalog</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos">70</span></a>
</span><span id="L-71"><a href="#L-71"><span class="linenos">71</span></a> <span class="k">if</span> <span class="n">isolate_tables</span><span class="p">:</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos">72</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">isolate_table_selects</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">)</span>
</span><span id="L-73"><a href="#L-73"><span class="linenos">73</span></a>
</span><span id="L-74"><a href="#L-74"><span class="linenos">74</span></a> <span class="k">if</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">PREFER_CTE_ALIAS_COLUMN</span><span class="p">:</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">pushdown_cte_alias_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos">76</span></a>
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a> <span class="k">if</span> <span class="n">qualify_columns</span><span class="p">:</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_columns_func</span><span class="p">(</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a> <span class="n">expression</span><span class="p">,</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a> <span class="n">schema</span><span class="p">,</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a> <span class="n">expand_alias_refs</span><span class="o">=</span><span class="n">expand_alias_refs</span><span class="p">,</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a> <span class="n">expand_stars</span><span class="o">=</span><span class="n">expand_stars</span><span class="p">,</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos">83</span></a> <span class="n">infer_schema</span><span class="o">=</span><span class="n">infer_schema</span><span class="p">,</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos">84</span></a> <span class="p">)</span>
</span><span id="L-85"><a href="#L-85"><span class="linenos">85</span></a>
</span><span id="L-86"><a href="#L-86"><span class="linenos">86</span></a> <span class="k">if</span> <span class="n">quote_identifiers</span><span class="p">:</span>
</span><span id="L-87"><a href="#L-87"><span class="linenos">87</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">quote_identifiers_func</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">identify</span><span class="o">=</span><span class="n">identify</span><span class="p">)</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos">88</span></a>
</span><span id="L-89"><a href="#L-89"><span class="linenos">89</span></a> <span class="k">if</span> <span class="n">validate_qualify_columns</span><span class="p">:</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos">90</span></a> <span class="n">validate_qualify_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos">91</span></a>
</span><span id="L-92"><a href="#L-92"><span class="linenos">92</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a> <span class="n">pushdown_cte_alias_columns</span> <span class="k">as</span> <span class="n">pushdown_cte_alias_columns_func</span><span class="p">,</span>
</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a> <span class="n">qualify_columns</span> <span class="k">as</span> <span class="n">qualify_columns_func</span><span class="p">,</span>
</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a> <span class="n">quote_identifiers</span> <span class="k">as</span> <span class="n">quote_identifiers_func</span><span class="p">,</span>
</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a> <span class="n">validate_qualify_columns</span> <span class="k">as</span> <span class="n">validate_qualify_columns_func</span><span class="p">,</span>
</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a><span class="p">)</span>
</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.qualify_tables</span> <span class="kn">import</span> <span class="n">qualify_tables</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="kn">from</span> <span class="nn">sqlglot.schema</span> <span class="kn">import</span> <span class="n">Schema</span><span class="p">,</span> <span class="n">ensure_schema</span>
</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a>
</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a>
</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a><span class="k">def</span> <span class="nf">qualify</span><span class="p">(</span>
</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">dict</span> <span class="o">|</span> <span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a> <span class="n">expand_alias_refs</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a> <span class="n">expand_stars</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a> <span class="n">infer_schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a> <span class="n">isolate_tables</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a> <span class="n">qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a> <span class="n">validate_qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a> <span class="n">quote_identifiers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a> <span class="n">identify</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a><span class="sd"> Rewrite sqlglot AST to have normalized and qualified tables and columns.</span>
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a>
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a><span class="sd"> This step is necessary for all further SQLGlot optimizations.</span>
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a>
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="sd"> Example:</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a><span class="sd"> &gt;&gt;&gt; schema = {&quot;tbl&quot;: {&quot;col&quot;: &quot;INT&quot;}}</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT col FROM tbl&quot;)</span>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a><span class="sd"> &gt;&gt;&gt; qualify(expression, schema=schema).sql()</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a><span class="sd"> &#39;SELECT &quot;tbl&quot;.&quot;col&quot; AS &quot;col&quot; FROM &quot;tbl&quot; AS &quot;tbl&quot;&#39;</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a><span class="sd"> Args:</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a><span class="sd"> expression: Expression to qualify.</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a><span class="sd"> db: Default database name for tables.</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a><span class="sd"> catalog: Default catalog name for tables.</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a><span class="sd"> schema: Schema to infer column names and types.</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a><span class="sd"> expand_alias_refs: Whether to expand references to aliases.</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a><span class="sd"> expand_stars: Whether to expand star queries. This is a necessary step</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a><span class="sd"> for most of the optimizer&#39;s rules to work; do not set to False unless you</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a><span class="sd"> know what you&#39;re doing!</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a><span class="sd"> infer_schema: Whether to infer the schema if missing.</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a><span class="sd"> isolate_tables: Whether to isolate table selects.</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a><span class="sd"> qualify_columns: Whether to qualify columns.</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a><span class="sd"> validate_qualify_columns: Whether to validate columns.</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a><span class="sd"> quote_identifiers: Whether to run the quote_identifiers step.</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a><span class="sd"> This step is necessary to ensure correctness for case sensitive queries.</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a><span class="sd"> But this flag is provided in case this step is performed at a later time.</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a><span class="sd"> identify: If True, quote all identifiers, else only necessary ones.</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a><span class="sd"> infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs&#39; schemas.</span>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a><span class="sd"> Returns:</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a><span class="sd"> The qualified expression.</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_tables</span><span class="p">(</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">expression</span><span class="p">,</span>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="n">db</span><span class="o">=</span><span class="n">db</span><span class="p">,</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="n">catalog</span><span class="o">=</span><span class="n">catalog</span><span class="p">,</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">,</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">infer_csv_schemas</span><span class="o">=</span><span class="n">infer_csv_schemas</span><span class="p">,</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="p">)</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">if</span> <span class="n">isolate_tables</span><span class="p">:</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">isolate_table_selects</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">)</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">if</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">PREFER_CTE_ALIAS_COLUMN</span><span class="p">:</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">pushdown_cte_alias_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">if</span> <span class="n">qualify_columns</span><span class="p">:</span>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_columns_func</span><span class="p">(</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="n">expression</span><span class="p">,</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">schema</span><span class="p">,</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="n">expand_alias_refs</span><span class="o">=</span><span class="n">expand_alias_refs</span><span class="p">,</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="n">expand_stars</span><span class="o">=</span><span class="n">expand_stars</span><span class="p">,</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">infer_schema</span><span class="o">=</span><span class="n">infer_schema</span><span class="p">,</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="p">)</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="k">if</span> <span class="n">quote_identifiers</span><span class="p">:</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">quote_identifiers_func</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">identify</span><span class="o">=</span><span class="n">identify</span><span class="p">)</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="k">if</span> <span class="n">validate_qualify_columns</span><span class="p">:</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">validate_qualify_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>
@ -157,86 +166,95 @@
<div class="attr function">
<span class="def">def</span>
<span class="name">qualify</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">,</span> <span class="n">Type</span><span class="p">[</span><span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">db</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">catalog</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">dict</span><span class="p">,</span> <span class="n"><a href="../schema.html#Schema">sqlglot.schema.Schema</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">expand_alias_refs</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">expand_stars</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">infer_schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">isolate_tables</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span>,</span><span class="param"> <span class="n">qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">validate_qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">quote_identifiers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">identify</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span></span><span class="return-annotation">) -> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span>:</span></span>
<span class="name">qualify</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">,</span> <span class="n">Type</span><span class="p">[</span><span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">db</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">catalog</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">dict</span><span class="p">,</span> <span class="n"><a href="../schema.html#Schema">sqlglot.schema.Schema</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">expand_alias_refs</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">expand_stars</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">infer_schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">isolate_tables</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span>,</span><span class="param"> <span class="n">qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">validate_qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">quote_identifiers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">identify</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span></span><span class="return-annotation">) -> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span>:</span></span>
<label class="view-source-button" for="qualify-view-source"><span>View Source</span></label>
</div>
<a class="headerlink" href="#qualify"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="qualify-20"><a href="#qualify-20"><span class="linenos">20</span></a><span class="k">def</span> <span class="nf">qualify</span><span class="p">(</span>
</span><span id="qualify-21"><a href="#qualify-21"><span class="linenos">21</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span>
</span><span id="qualify-22"><a href="#qualify-22"><span class="linenos">22</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify-23"><a href="#qualify-23"><span class="linenos">23</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify-24"><a href="#qualify-24"><span class="linenos">24</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify-25"><a href="#qualify-25"><span class="linenos">25</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">dict</span> <span class="o">|</span> <span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify-26"><a href="#qualify-26"><span class="linenos">26</span></a> <span class="n">expand_alias_refs</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="qualify-27"><a href="#qualify-27"><span class="linenos">27</span></a> <span class="n">expand_stars</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="qualify-28"><a href="#qualify-28"><span class="linenos">28</span></a> <span class="n">infer_schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify-29"><a href="#qualify-29"><span class="linenos">29</span></a> <span class="n">isolate_tables</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="qualify-30"><a href="#qualify-30"><span class="linenos">30</span></a> <span class="n">qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="qualify-31"><a href="#qualify-31"><span class="linenos">31</span></a> <span class="n">validate_qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="qualify-32"><a href="#qualify-32"><span class="linenos">32</span></a> <span class="n">quote_identifiers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="qualify-33"><a href="#qualify-33"><span class="linenos">33</span></a> <span class="n">identify</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="qualify-34"><a href="#qualify-34"><span class="linenos">34</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="qualify-35"><a href="#qualify-35"><span class="linenos">35</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="qualify-36"><a href="#qualify-36"><span class="linenos">36</span></a><span class="sd"> Rewrite sqlglot AST to have normalized and qualified tables and columns.</span>
</span><span id="qualify-37"><a href="#qualify-37"><span class="linenos">37</span></a>
</span><span id="qualify-38"><a href="#qualify-38"><span class="linenos">38</span></a><span class="sd"> This step is necessary for all further SQLGlot optimizations.</span>
</span><span id="qualify-39"><a href="#qualify-39"><span class="linenos">39</span></a>
</span><span id="qualify-40"><a href="#qualify-40"><span class="linenos">40</span></a><span class="sd"> Example:</span>
</span><span id="qualify-41"><a href="#qualify-41"><span class="linenos">41</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="qualify-42"><a href="#qualify-42"><span class="linenos">42</span></a><span class="sd"> &gt;&gt;&gt; schema = {&quot;tbl&quot;: {&quot;col&quot;: &quot;INT&quot;}}</span>
</span><span id="qualify-43"><a href="#qualify-43"><span class="linenos">43</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT col FROM tbl&quot;)</span>
</span><span id="qualify-44"><a href="#qualify-44"><span class="linenos">44</span></a><span class="sd"> &gt;&gt;&gt; qualify(expression, schema=schema).sql()</span>
</span><span id="qualify-45"><a href="#qualify-45"><span class="linenos">45</span></a><span class="sd"> &#39;SELECT &quot;tbl&quot;.&quot;col&quot; AS &quot;col&quot; FROM &quot;tbl&quot; AS &quot;tbl&quot;&#39;</span>
</span><span id="qualify-46"><a href="#qualify-46"><span class="linenos">46</span></a>
</span><span id="qualify-47"><a href="#qualify-47"><span class="linenos">47</span></a><span class="sd"> Args:</span>
</span><span id="qualify-48"><a href="#qualify-48"><span class="linenos">48</span></a><span class="sd"> expression: Expression to qualify.</span>
</span><span id="qualify-49"><a href="#qualify-49"><span class="linenos">49</span></a><span class="sd"> db: Default database name for tables.</span>
</span><span id="qualify-50"><a href="#qualify-50"><span class="linenos">50</span></a><span class="sd"> catalog: Default catalog name for tables.</span>
</span><span id="qualify-51"><a href="#qualify-51"><span class="linenos">51</span></a><span class="sd"> schema: Schema to infer column names and types.</span>
</span><span id="qualify-52"><a href="#qualify-52"><span class="linenos">52</span></a><span class="sd"> expand_alias_refs: Whether to expand references to aliases.</span>
</span><span id="qualify-53"><a href="#qualify-53"><span class="linenos">53</span></a><span class="sd"> expand_stars: Whether to expand star queries. This is a necessary step</span>
</span><span id="qualify-54"><a href="#qualify-54"><span class="linenos">54</span></a><span class="sd"> for most of the optimizer&#39;s rules to work; do not set to False unless you</span>
</span><span id="qualify-55"><a href="#qualify-55"><span class="linenos">55</span></a><span class="sd"> know what you&#39;re doing!</span>
</span><span id="qualify-56"><a href="#qualify-56"><span class="linenos">56</span></a><span class="sd"> infer_schema: Whether to infer the schema if missing.</span>
</span><span id="qualify-57"><a href="#qualify-57"><span class="linenos">57</span></a><span class="sd"> isolate_tables: Whether to isolate table selects.</span>
</span><span id="qualify-58"><a href="#qualify-58"><span class="linenos">58</span></a><span class="sd"> qualify_columns: Whether to qualify columns.</span>
</span><span id="qualify-59"><a href="#qualify-59"><span class="linenos">59</span></a><span class="sd"> validate_qualify_columns: Whether to validate columns.</span>
</span><span id="qualify-60"><a href="#qualify-60"><span class="linenos">60</span></a><span class="sd"> quote_identifiers: Whether to run the quote_identifiers step.</span>
</span><span id="qualify-61"><a href="#qualify-61"><span class="linenos">61</span></a><span class="sd"> This step is necessary to ensure correctness for case sensitive queries.</span>
</span><span id="qualify-62"><a href="#qualify-62"><span class="linenos">62</span></a><span class="sd"> But this flag is provided in case this step is performed at a later time.</span>
</span><span id="qualify-63"><a href="#qualify-63"><span class="linenos">63</span></a><span class="sd"> identify: If True, quote all identifiers, else only necessary ones.</span>
</span><span id="qualify-64"><a href="#qualify-64"><span class="linenos">64</span></a>
</span><span id="qualify-65"><a href="#qualify-65"><span class="linenos">65</span></a><span class="sd"> Returns:</span>
</span><span id="qualify-66"><a href="#qualify-66"><span class="linenos">66</span></a><span class="sd"> The qualified expression.</span>
</span><span id="qualify-67"><a href="#qualify-67"><span class="linenos">67</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="qualify-68"><a href="#qualify-68"><span class="linenos">68</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="qualify-69"><a href="#qualify-69"><span class="linenos">69</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="qualify-70"><a href="#qualify-70"><span class="linenos">70</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_tables</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">db</span><span class="o">=</span><span class="n">db</span><span class="p">,</span> <span class="n">catalog</span><span class="o">=</span><span class="n">catalog</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="qualify-71"><a href="#qualify-71"><span class="linenos">71</span></a>
</span><span id="qualify-72"><a href="#qualify-72"><span class="linenos">72</span></a> <span class="k">if</span> <span class="n">isolate_tables</span><span class="p">:</span>
</span><span id="qualify-73"><a href="#qualify-73"><span class="linenos">73</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">isolate_table_selects</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">)</span>
</span><span id="qualify-74"><a href="#qualify-74"><span class="linenos">74</span></a>
</span><span id="qualify-75"><a href="#qualify-75"><span class="linenos">75</span></a> <span class="k">if</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">PREFER_CTE_ALIAS_COLUMN</span><span class="p">:</span>
</span><span id="qualify-76"><a href="#qualify-76"><span class="linenos">76</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">pushdown_cte_alias_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="qualify-77"><a href="#qualify-77"><span class="linenos">77</span></a>
</span><span id="qualify-78"><a href="#qualify-78"><span class="linenos">78</span></a> <span class="k">if</span> <span class="n">qualify_columns</span><span class="p">:</span>
</span><span id="qualify-79"><a href="#qualify-79"><span class="linenos">79</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_columns_func</span><span class="p">(</span>
</span><span id="qualify-80"><a href="#qualify-80"><span class="linenos">80</span></a> <span class="n">expression</span><span class="p">,</span>
</span><span id="qualify-81"><a href="#qualify-81"><span class="linenos">81</span></a> <span class="n">schema</span><span class="p">,</span>
</span><span id="qualify-82"><a href="#qualify-82"><span class="linenos">82</span></a> <span class="n">expand_alias_refs</span><span class="o">=</span><span class="n">expand_alias_refs</span><span class="p">,</span>
</span><span id="qualify-83"><a href="#qualify-83"><span class="linenos">83</span></a> <span class="n">expand_stars</span><span class="o">=</span><span class="n">expand_stars</span><span class="p">,</span>
</span><span id="qualify-84"><a href="#qualify-84"><span class="linenos">84</span></a> <span class="n">infer_schema</span><span class="o">=</span><span class="n">infer_schema</span><span class="p">,</span>
</span><span id="qualify-85"><a href="#qualify-85"><span class="linenos">85</span></a> <span class="p">)</span>
</span><span id="qualify-86"><a href="#qualify-86"><span class="linenos">86</span></a>
</span><span id="qualify-87"><a href="#qualify-87"><span class="linenos">87</span></a> <span class="k">if</span> <span class="n">quote_identifiers</span><span class="p">:</span>
</span><span id="qualify-88"><a href="#qualify-88"><span class="linenos">88</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">quote_identifiers_func</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">identify</span><span class="o">=</span><span class="n">identify</span><span class="p">)</span>
</span><span id="qualify-89"><a href="#qualify-89"><span class="linenos">89</span></a>
</span><span id="qualify-90"><a href="#qualify-90"><span class="linenos">90</span></a> <span class="k">if</span> <span class="n">validate_qualify_columns</span><span class="p">:</span>
</span><span id="qualify-91"><a href="#qualify-91"><span class="linenos">91</span></a> <span class="n">validate_qualify_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="qualify-92"><a href="#qualify-92"><span class="linenos">92</span></a>
</span><span id="qualify-93"><a href="#qualify-93"><span class="linenos">93</span></a> <span class="k">return</span> <span class="n">expression</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="qualify-20"><a href="#qualify-20"><span class="linenos"> 20</span></a><span class="k">def</span> <span class="nf">qualify</span><span class="p">(</span>
</span><span id="qualify-21"><a href="#qualify-21"><span class="linenos"> 21</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span>
</span><span id="qualify-22"><a href="#qualify-22"><span class="linenos"> 22</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify-23"><a href="#qualify-23"><span class="linenos"> 23</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify-24"><a href="#qualify-24"><span class="linenos"> 24</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify-25"><a href="#qualify-25"><span class="linenos"> 25</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">dict</span> <span class="o">|</span> <span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify-26"><a href="#qualify-26"><span class="linenos"> 26</span></a> <span class="n">expand_alias_refs</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="qualify-27"><a href="#qualify-27"><span class="linenos"> 27</span></a> <span class="n">expand_stars</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="qualify-28"><a href="#qualify-28"><span class="linenos"> 28</span></a> <span class="n">infer_schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">bool</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify-29"><a href="#qualify-29"><span class="linenos"> 29</span></a> <span class="n">isolate_tables</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="qualify-30"><a href="#qualify-30"><span class="linenos"> 30</span></a> <span class="n">qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="qualify-31"><a href="#qualify-31"><span class="linenos"> 31</span></a> <span class="n">validate_qualify_columns</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="qualify-32"><a href="#qualify-32"><span class="linenos"> 32</span></a> <span class="n">quote_identifiers</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="qualify-33"><a href="#qualify-33"><span class="linenos"> 33</span></a> <span class="n">identify</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="qualify-34"><a href="#qualify-34"><span class="linenos"> 34</span></a> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="qualify-35"><a href="#qualify-35"><span class="linenos"> 35</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
</span><span id="qualify-36"><a href="#qualify-36"><span class="linenos"> 36</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="qualify-37"><a href="#qualify-37"><span class="linenos"> 37</span></a><span class="sd"> Rewrite sqlglot AST to have normalized and qualified tables and columns.</span>
</span><span id="qualify-38"><a href="#qualify-38"><span class="linenos"> 38</span></a>
</span><span id="qualify-39"><a href="#qualify-39"><span class="linenos"> 39</span></a><span class="sd"> This step is necessary for all further SQLGlot optimizations.</span>
</span><span id="qualify-40"><a href="#qualify-40"><span class="linenos"> 40</span></a>
</span><span id="qualify-41"><a href="#qualify-41"><span class="linenos"> 41</span></a><span class="sd"> Example:</span>
</span><span id="qualify-42"><a href="#qualify-42"><span class="linenos"> 42</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="qualify-43"><a href="#qualify-43"><span class="linenos"> 43</span></a><span class="sd"> &gt;&gt;&gt; schema = {&quot;tbl&quot;: {&quot;col&quot;: &quot;INT&quot;}}</span>
</span><span id="qualify-44"><a href="#qualify-44"><span class="linenos"> 44</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT col FROM tbl&quot;)</span>
</span><span id="qualify-45"><a href="#qualify-45"><span class="linenos"> 45</span></a><span class="sd"> &gt;&gt;&gt; qualify(expression, schema=schema).sql()</span>
</span><span id="qualify-46"><a href="#qualify-46"><span class="linenos"> 46</span></a><span class="sd"> &#39;SELECT &quot;tbl&quot;.&quot;col&quot; AS &quot;col&quot; FROM &quot;tbl&quot; AS &quot;tbl&quot;&#39;</span>
</span><span id="qualify-47"><a href="#qualify-47"><span class="linenos"> 47</span></a>
</span><span id="qualify-48"><a href="#qualify-48"><span class="linenos"> 48</span></a><span class="sd"> Args:</span>
</span><span id="qualify-49"><a href="#qualify-49"><span class="linenos"> 49</span></a><span class="sd"> expression: Expression to qualify.</span>
</span><span id="qualify-50"><a href="#qualify-50"><span class="linenos"> 50</span></a><span class="sd"> db: Default database name for tables.</span>
</span><span id="qualify-51"><a href="#qualify-51"><span class="linenos"> 51</span></a><span class="sd"> catalog: Default catalog name for tables.</span>
</span><span id="qualify-52"><a href="#qualify-52"><span class="linenos"> 52</span></a><span class="sd"> schema: Schema to infer column names and types.</span>
</span><span id="qualify-53"><a href="#qualify-53"><span class="linenos"> 53</span></a><span class="sd"> expand_alias_refs: Whether to expand references to aliases.</span>
</span><span id="qualify-54"><a href="#qualify-54"><span class="linenos"> 54</span></a><span class="sd"> expand_stars: Whether to expand star queries. This is a necessary step</span>
</span><span id="qualify-55"><a href="#qualify-55"><span class="linenos"> 55</span></a><span class="sd"> for most of the optimizer&#39;s rules to work; do not set to False unless you</span>
</span><span id="qualify-56"><a href="#qualify-56"><span class="linenos"> 56</span></a><span class="sd"> know what you&#39;re doing!</span>
</span><span id="qualify-57"><a href="#qualify-57"><span class="linenos"> 57</span></a><span class="sd"> infer_schema: Whether to infer the schema if missing.</span>
</span><span id="qualify-58"><a href="#qualify-58"><span class="linenos"> 58</span></a><span class="sd"> isolate_tables: Whether to isolate table selects.</span>
</span><span id="qualify-59"><a href="#qualify-59"><span class="linenos"> 59</span></a><span class="sd"> qualify_columns: Whether to qualify columns.</span>
</span><span id="qualify-60"><a href="#qualify-60"><span class="linenos"> 60</span></a><span class="sd"> validate_qualify_columns: Whether to validate columns.</span>
</span><span id="qualify-61"><a href="#qualify-61"><span class="linenos"> 61</span></a><span class="sd"> quote_identifiers: Whether to run the quote_identifiers step.</span>
</span><span id="qualify-62"><a href="#qualify-62"><span class="linenos"> 62</span></a><span class="sd"> This step is necessary to ensure correctness for case sensitive queries.</span>
</span><span id="qualify-63"><a href="#qualify-63"><span class="linenos"> 63</span></a><span class="sd"> But this flag is provided in case this step is performed at a later time.</span>
</span><span id="qualify-64"><a href="#qualify-64"><span class="linenos"> 64</span></a><span class="sd"> identify: If True, quote all identifiers, else only necessary ones.</span>
</span><span id="qualify-65"><a href="#qualify-65"><span class="linenos"> 65</span></a><span class="sd"> infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs&#39; schemas.</span>
</span><span id="qualify-66"><a href="#qualify-66"><span class="linenos"> 66</span></a>
</span><span id="qualify-67"><a href="#qualify-67"><span class="linenos"> 67</span></a><span class="sd"> Returns:</span>
</span><span id="qualify-68"><a href="#qualify-68"><span class="linenos"> 68</span></a><span class="sd"> The qualified expression.</span>
</span><span id="qualify-69"><a href="#qualify-69"><span class="linenos"> 69</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="qualify-70"><a href="#qualify-70"><span class="linenos"> 70</span></a> <span class="n">schema</span> <span class="o">=</span> <span class="n">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="qualify-71"><a href="#qualify-71"><span class="linenos"> 71</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="qualify-72"><a href="#qualify-72"><span class="linenos"> 72</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_tables</span><span class="p">(</span>
</span><span id="qualify-73"><a href="#qualify-73"><span class="linenos"> 73</span></a> <span class="n">expression</span><span class="p">,</span>
</span><span id="qualify-74"><a href="#qualify-74"><span class="linenos"> 74</span></a> <span class="n">db</span><span class="o">=</span><span class="n">db</span><span class="p">,</span>
</span><span id="qualify-75"><a href="#qualify-75"><span class="linenos"> 75</span></a> <span class="n">catalog</span><span class="o">=</span><span class="n">catalog</span><span class="p">,</span>
</span><span id="qualify-76"><a href="#qualify-76"><span class="linenos"> 76</span></a> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">,</span>
</span><span id="qualify-77"><a href="#qualify-77"><span class="linenos"> 77</span></a> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span>
</span><span id="qualify-78"><a href="#qualify-78"><span class="linenos"> 78</span></a> <span class="n">infer_csv_schemas</span><span class="o">=</span><span class="n">infer_csv_schemas</span><span class="p">,</span>
</span><span id="qualify-79"><a href="#qualify-79"><span class="linenos"> 79</span></a> <span class="p">)</span>
</span><span id="qualify-80"><a href="#qualify-80"><span class="linenos"> 80</span></a>
</span><span id="qualify-81"><a href="#qualify-81"><span class="linenos"> 81</span></a> <span class="k">if</span> <span class="n">isolate_tables</span><span class="p">:</span>
</span><span id="qualify-82"><a href="#qualify-82"><span class="linenos"> 82</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">isolate_table_selects</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">schema</span><span class="p">)</span>
</span><span id="qualify-83"><a href="#qualify-83"><span class="linenos"> 83</span></a>
</span><span id="qualify-84"><a href="#qualify-84"><span class="linenos"> 84</span></a> <span class="k">if</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">PREFER_CTE_ALIAS_COLUMN</span><span class="p">:</span>
</span><span id="qualify-85"><a href="#qualify-85"><span class="linenos"> 85</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">pushdown_cte_alias_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="qualify-86"><a href="#qualify-86"><span class="linenos"> 86</span></a>
</span><span id="qualify-87"><a href="#qualify-87"><span class="linenos"> 87</span></a> <span class="k">if</span> <span class="n">qualify_columns</span><span class="p">:</span>
</span><span id="qualify-88"><a href="#qualify-88"><span class="linenos"> 88</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">qualify_columns_func</span><span class="p">(</span>
</span><span id="qualify-89"><a href="#qualify-89"><span class="linenos"> 89</span></a> <span class="n">expression</span><span class="p">,</span>
</span><span id="qualify-90"><a href="#qualify-90"><span class="linenos"> 90</span></a> <span class="n">schema</span><span class="p">,</span>
</span><span id="qualify-91"><a href="#qualify-91"><span class="linenos"> 91</span></a> <span class="n">expand_alias_refs</span><span class="o">=</span><span class="n">expand_alias_refs</span><span class="p">,</span>
</span><span id="qualify-92"><a href="#qualify-92"><span class="linenos"> 92</span></a> <span class="n">expand_stars</span><span class="o">=</span><span class="n">expand_stars</span><span class="p">,</span>
</span><span id="qualify-93"><a href="#qualify-93"><span class="linenos"> 93</span></a> <span class="n">infer_schema</span><span class="o">=</span><span class="n">infer_schema</span><span class="p">,</span>
</span><span id="qualify-94"><a href="#qualify-94"><span class="linenos"> 94</span></a> <span class="p">)</span>
</span><span id="qualify-95"><a href="#qualify-95"><span class="linenos"> 95</span></a>
</span><span id="qualify-96"><a href="#qualify-96"><span class="linenos"> 96</span></a> <span class="k">if</span> <span class="n">quote_identifiers</span><span class="p">:</span>
</span><span id="qualify-97"><a href="#qualify-97"><span class="linenos"> 97</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">quote_identifiers_func</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">identify</span><span class="o">=</span><span class="n">identify</span><span class="p">)</span>
</span><span id="qualify-98"><a href="#qualify-98"><span class="linenos"> 98</span></a>
</span><span id="qualify-99"><a href="#qualify-99"><span class="linenos"> 99</span></a> <span class="k">if</span> <span class="n">validate_qualify_columns</span><span class="p">:</span>
</span><span id="qualify-100"><a href="#qualify-100"><span class="linenos">100</span></a> <span class="n">validate_qualify_columns_func</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="qualify-101"><a href="#qualify-101"><span class="linenos">101</span></a>
</span><span id="qualify-102"><a href="#qualify-102"><span class="linenos">102</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>
@ -276,6 +294,7 @@ know what you're doing!</li>
This step is necessary to ensure correctness for case sensitive queries.
But this flag is provided in case this step is performed at a later time.</li>
<li><strong>identify:</strong> If True, quote all identifiers, else only necessary ones.</li>
<li><strong>infer_csv_schemas:</strong> Whether to scan READ_CSV calls in order to infer the CSVs' schemas.</li>
</ul>
<h6 id="returns">Returns:</h6>

File diff suppressed because it is too large Load diff

View file

@ -76,131 +76,133 @@
</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span>
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span>
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span>
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a>
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a><span class="sd"> Examples:</span>
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM tbl&quot;)</span>
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression, db=&quot;db&quot;).sql()</span>
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a><span class="sd"> &#39;SELECT 1 FROM db.tbl AS tbl&#39;</span>
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a><span class="sd"> &gt;&gt;&gt;</span>
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM (t1 JOIN t2) AS t&quot;)</span>
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression).sql()</span>
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a><span class="sd"> &#39;SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t&#39;</span>
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a>
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a><span class="sd"> Args:</span>
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a><span class="sd"> expression: Expression to qualify</span>
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a><span class="sd"> db: Database name</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="sd"> catalog: Catalog name</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a><span class="sd"> schema: A schema to populate</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a><span class="sd"> dialect: The dialect to parse catalog and schema into.</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a><span class="sd"> Returns:</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a><span class="sd"> The qualified expression.</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">&quot;_q_&quot;</span><span class="p">)</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="n">db</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">db</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">catalog</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">catalog</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="k">def</span> <span class="nf">_qualify</span><span class="p">(</span><span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">,</span> <span class="n">db</span><span class="p">)</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">)</span> <span class="ow">and</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">,</span> <span class="n">catalog</span><span class="p">)</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="k">if</span> <span class="p">(</span><span class="n">db</span> <span class="ow">or</span> <span class="n">catalog</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">)):</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">&quot;*&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">):</span>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">table_aliases</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span>
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span>
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span>
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a>
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a><span class="sd"> Examples:</span>
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM tbl&quot;)</span>
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression, db=&quot;db&quot;).sql()</span>
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a><span class="sd"> &#39;SELECT 1 FROM db.tbl AS tbl&#39;</span>
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a><span class="sd"> &gt;&gt;&gt;</span>
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM (t1 JOIN t2) AS t&quot;)</span>
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression).sql()</span>
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a><span class="sd"> &#39;SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t&#39;</span>
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a>
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a><span class="sd"> Args:</span>
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a><span class="sd"> expression: Expression to qualify</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="sd"> db: Database name</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a><span class="sd"> catalog: Catalog name</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a><span class="sd"> schema: A schema to populate</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a><span class="sd"> infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs&#39; schemas.</span>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a><span class="sd"> dialect: The dialect to parse catalog and schema into.</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a><span class="sd"> Returns:</span>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a><span class="sd"> The qualified expression.</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">&quot;_q_&quot;</span><span class="p">)</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="n">db</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">db</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">catalog</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">catalog</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="k">def</span> <span class="nf">_qualify</span><span class="p">(</span><span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">,</span> <span class="n">db</span><span class="p">)</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">)</span> <span class="ow">and</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">,</span> <span class="n">catalog</span><span class="p">)</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="k">if</span> <span class="p">(</span><span class="n">db</span> <span class="ow">or</span> <span class="n">catalog</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">)):</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">&quot;*&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">):</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="c1"># Don&#39;t add the pivot&#39;s alias to the pivoted table, use the table&#39;s name instead</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span> <span class="o">==</span> <span class="n">name</span><span class="p">:</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">table_aliases</span><span class="p">[</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">parts</span><span class="p">)]</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">source</span><span class="o">.</span><span class="n">alias</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="p">)</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="n">table_aliases</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="c1"># Don&#39;t add the pivot&#39;s alias to the pivoted table, use the table&#39;s name instead</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span> <span class="o">==</span> <span class="n">name</span><span class="p">:</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">table_aliases</span><span class="p">[</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">parts</span><span class="p">)]</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">source</span><span class="o">.</span><span class="n">alias</span>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="p">)</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="p">)</span>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="k">if</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">source</span><span class="p">,</span>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="p">)</span>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="p">)</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">))</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="p">):</span>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;table&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="p">)</span>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">if</span> <span class="n">infer_csv_schemas</span> <span class="ow">and</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="n">source</span><span class="p">,</span>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="p">)</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="p">)</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">))</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="p">):</span>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;table&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>
@ -210,7 +212,7 @@
<div class="attr function">
<span class="def">def</span>
<span class="name">qualify_tables</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="o">~</span><span class="n">E</span>,</span><span class="param"> <span class="n">db</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n"><a href="../expressions.html#Identifier">sqlglot.expressions.Identifier</a></span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">catalog</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n"><a href="../expressions.html#Identifier">sqlglot.expressions.Identifier</a></span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n"><a href="../schema.html#Schema">sqlglot.schema.Schema</a></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">,</span> <span class="n">Type</span><span class="p">[</span><span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="o">~</span><span class="n">E</span>:</span></span>
<span class="name">qualify_tables</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="o">~</span><span class="n">E</span>,</span><span class="param"> <span class="n">db</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n"><a href="../expressions.html#Identifier">sqlglot.expressions.Identifier</a></span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">catalog</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n"><a href="../expressions.html#Identifier">sqlglot.expressions.Identifier</a></span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n"><a href="../schema.html#Schema">sqlglot.schema.Schema</a></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">,</span> <span class="n">Type</span><span class="p">[</span><span class="n"><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="o">~</span><span class="n">E</span>:</span></span>
<label class="view-source-button" for="qualify_tables-view-source"><span>View Source</span></label>
@ -221,131 +223,133 @@
</span><span id="qualify_tables-19"><a href="#qualify_tables-19"><span class="linenos"> 19</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify_tables-20"><a href="#qualify_tables-20"><span class="linenos"> 20</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify_tables-21"><a href="#qualify_tables-21"><span class="linenos"> 21</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify_tables-22"><a href="#qualify_tables-22"><span class="linenos"> 22</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify_tables-23"><a href="#qualify_tables-23"><span class="linenos"> 23</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span>
</span><span id="qualify_tables-24"><a href="#qualify_tables-24"><span class="linenos"> 24</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="qualify_tables-25"><a href="#qualify_tables-25"><span class="linenos"> 25</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span>
</span><span id="qualify_tables-26"><a href="#qualify_tables-26"><span class="linenos"> 26</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span>
</span><span id="qualify_tables-27"><a href="#qualify_tables-27"><span class="linenos"> 27</span></a>
</span><span id="qualify_tables-28"><a href="#qualify_tables-28"><span class="linenos"> 28</span></a><span class="sd"> Examples:</span>
</span><span id="qualify_tables-29"><a href="#qualify_tables-29"><span class="linenos"> 29</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="qualify_tables-30"><a href="#qualify_tables-30"><span class="linenos"> 30</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM tbl&quot;)</span>
</span><span id="qualify_tables-31"><a href="#qualify_tables-31"><span class="linenos"> 31</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression, db=&quot;db&quot;).sql()</span>
</span><span id="qualify_tables-32"><a href="#qualify_tables-32"><span class="linenos"> 32</span></a><span class="sd"> &#39;SELECT 1 FROM db.tbl AS tbl&#39;</span>
</span><span id="qualify_tables-33"><a href="#qualify_tables-33"><span class="linenos"> 33</span></a><span class="sd"> &gt;&gt;&gt;</span>
</span><span id="qualify_tables-34"><a href="#qualify_tables-34"><span class="linenos"> 34</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM (t1 JOIN t2) AS t&quot;)</span>
</span><span id="qualify_tables-35"><a href="#qualify_tables-35"><span class="linenos"> 35</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression).sql()</span>
</span><span id="qualify_tables-36"><a href="#qualify_tables-36"><span class="linenos"> 36</span></a><span class="sd"> &#39;SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t&#39;</span>
</span><span id="qualify_tables-37"><a href="#qualify_tables-37"><span class="linenos"> 37</span></a>
</span><span id="qualify_tables-38"><a href="#qualify_tables-38"><span class="linenos"> 38</span></a><span class="sd"> Args:</span>
</span><span id="qualify_tables-39"><a href="#qualify_tables-39"><span class="linenos"> 39</span></a><span class="sd"> expression: Expression to qualify</span>
</span><span id="qualify_tables-40"><a href="#qualify_tables-40"><span class="linenos"> 40</span></a><span class="sd"> db: Database name</span>
</span><span id="qualify_tables-41"><a href="#qualify_tables-41"><span class="linenos"> 41</span></a><span class="sd"> catalog: Catalog name</span>
</span><span id="qualify_tables-42"><a href="#qualify_tables-42"><span class="linenos"> 42</span></a><span class="sd"> schema: A schema to populate</span>
</span><span id="qualify_tables-43"><a href="#qualify_tables-43"><span class="linenos"> 43</span></a><span class="sd"> dialect: The dialect to parse catalog and schema into.</span>
</span><span id="qualify_tables-44"><a href="#qualify_tables-44"><span class="linenos"> 44</span></a>
</span><span id="qualify_tables-45"><a href="#qualify_tables-45"><span class="linenos"> 45</span></a><span class="sd"> Returns:</span>
</span><span id="qualify_tables-46"><a href="#qualify_tables-46"><span class="linenos"> 46</span></a><span class="sd"> The qualified expression.</span>
</span><span id="qualify_tables-47"><a href="#qualify_tables-47"><span class="linenos"> 47</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="qualify_tables-48"><a href="#qualify_tables-48"><span class="linenos"> 48</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">&quot;_q_&quot;</span><span class="p">)</span>
</span><span id="qualify_tables-49"><a href="#qualify_tables-49"><span class="linenos"> 49</span></a> <span class="n">db</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">db</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="qualify_tables-50"><a href="#qualify_tables-50"><span class="linenos"> 50</span></a> <span class="n">catalog</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">catalog</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="qualify_tables-51"><a href="#qualify_tables-51"><span class="linenos"> 51</span></a>
</span><span id="qualify_tables-52"><a href="#qualify_tables-52"><span class="linenos"> 52</span></a> <span class="k">def</span> <span class="nf">_qualify</span><span class="p">(</span><span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="qualify_tables-53"><a href="#qualify_tables-53"><span class="linenos"> 53</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
</span><span id="qualify_tables-54"><a href="#qualify_tables-54"><span class="linenos"> 54</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="qualify_tables-55"><a href="#qualify_tables-55"><span class="linenos"> 55</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">,</span> <span class="n">db</span><span class="p">)</span>
</span><span id="qualify_tables-56"><a href="#qualify_tables-56"><span class="linenos"> 56</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">)</span> <span class="ow">and</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="qualify_tables-57"><a href="#qualify_tables-57"><span class="linenos"> 57</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">,</span> <span class="n">catalog</span><span class="p">)</span>
</span><span id="qualify_tables-58"><a href="#qualify_tables-58"><span class="linenos"> 58</span></a>
</span><span id="qualify_tables-59"><a href="#qualify_tables-59"><span class="linenos"> 59</span></a> <span class="k">if</span> <span class="p">(</span><span class="n">db</span> <span class="ow">or</span> <span class="n">catalog</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span>
</span><span id="qualify_tables-60"><a href="#qualify_tables-60"><span class="linenos"> 60</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">)):</span>
</span><span id="qualify_tables-61"><a href="#qualify_tables-61"><span class="linenos"> 61</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="qualify_tables-62"><a href="#qualify_tables-62"><span class="linenos"> 62</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="qualify_tables-63"><a href="#qualify_tables-63"><span class="linenos"> 63</span></a>
</span><span id="qualify_tables-64"><a href="#qualify_tables-64"><span class="linenos"> 64</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
</span><span id="qualify_tables-65"><a href="#qualify_tables-65"><span class="linenos"> 65</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
</span><span id="qualify_tables-66"><a href="#qualify_tables-66"><span class="linenos"> 66</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
</span><span id="qualify_tables-67"><a href="#qualify_tables-67"><span class="linenos"> 67</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
</span><span id="qualify_tables-68"><a href="#qualify_tables-68"><span class="linenos"> 68</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="qualify_tables-69"><a href="#qualify_tables-69"><span class="linenos"> 69</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="qualify_tables-70"><a href="#qualify_tables-70"><span class="linenos"> 70</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">&quot;*&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
</span><span id="qualify_tables-71"><a href="#qualify_tables-71"><span class="linenos"> 71</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span>
</span><span id="qualify_tables-72"><a href="#qualify_tables-72"><span class="linenos"> 72</span></a>
</span><span id="qualify_tables-73"><a href="#qualify_tables-73"><span class="linenos"> 73</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">):</span>
</span><span id="qualify_tables-74"><a href="#qualify_tables-74"><span class="linenos"> 74</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
</span><span id="qualify_tables-75"><a href="#qualify_tables-75"><span class="linenos"> 75</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
</span><span id="qualify_tables-76"><a href="#qualify_tables-76"><span class="linenos"> 76</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
</span><span id="qualify_tables-77"><a href="#qualify_tables-77"><span class="linenos"> 77</span></a>
</span><span id="qualify_tables-78"><a href="#qualify_tables-78"><span class="linenos"> 78</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="qualify_tables-79"><a href="#qualify_tables-79"><span class="linenos"> 79</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="qualify_tables-80"><a href="#qualify_tables-80"><span class="linenos"> 80</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
</span><span id="qualify_tables-81"><a href="#qualify_tables-81"><span class="linenos"> 81</span></a>
</span><span id="qualify_tables-82"><a href="#qualify_tables-82"><span class="linenos"> 82</span></a> <span class="n">table_aliases</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="qualify_tables-22"><a href="#qualify_tables-22"><span class="linenos"> 22</span></a> <span class="n">infer_csv_schemas</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="qualify_tables-23"><a href="#qualify_tables-23"><span class="linenos"> 23</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="qualify_tables-24"><a href="#qualify_tables-24"><span class="linenos"> 24</span></a><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span>
</span><span id="qualify_tables-25"><a href="#qualify_tables-25"><span class="linenos"> 25</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="qualify_tables-26"><a href="#qualify_tables-26"><span class="linenos"> 26</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span>
</span><span id="qualify_tables-27"><a href="#qualify_tables-27"><span class="linenos"> 27</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span>
</span><span id="qualify_tables-28"><a href="#qualify_tables-28"><span class="linenos"> 28</span></a>
</span><span id="qualify_tables-29"><a href="#qualify_tables-29"><span class="linenos"> 29</span></a><span class="sd"> Examples:</span>
</span><span id="qualify_tables-30"><a href="#qualify_tables-30"><span class="linenos"> 30</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="qualify_tables-31"><a href="#qualify_tables-31"><span class="linenos"> 31</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM tbl&quot;)</span>
</span><span id="qualify_tables-32"><a href="#qualify_tables-32"><span class="linenos"> 32</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression, db=&quot;db&quot;).sql()</span>
</span><span id="qualify_tables-33"><a href="#qualify_tables-33"><span class="linenos"> 33</span></a><span class="sd"> &#39;SELECT 1 FROM db.tbl AS tbl&#39;</span>
</span><span id="qualify_tables-34"><a href="#qualify_tables-34"><span class="linenos"> 34</span></a><span class="sd"> &gt;&gt;&gt;</span>
</span><span id="qualify_tables-35"><a href="#qualify_tables-35"><span class="linenos"> 35</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;SELECT 1 FROM (t1 JOIN t2) AS t&quot;)</span>
</span><span id="qualify_tables-36"><a href="#qualify_tables-36"><span class="linenos"> 36</span></a><span class="sd"> &gt;&gt;&gt; qualify_tables(expression).sql()</span>
</span><span id="qualify_tables-37"><a href="#qualify_tables-37"><span class="linenos"> 37</span></a><span class="sd"> &#39;SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t&#39;</span>
</span><span id="qualify_tables-38"><a href="#qualify_tables-38"><span class="linenos"> 38</span></a>
</span><span id="qualify_tables-39"><a href="#qualify_tables-39"><span class="linenos"> 39</span></a><span class="sd"> Args:</span>
</span><span id="qualify_tables-40"><a href="#qualify_tables-40"><span class="linenos"> 40</span></a><span class="sd"> expression: Expression to qualify</span>
</span><span id="qualify_tables-41"><a href="#qualify_tables-41"><span class="linenos"> 41</span></a><span class="sd"> db: Database name</span>
</span><span id="qualify_tables-42"><a href="#qualify_tables-42"><span class="linenos"> 42</span></a><span class="sd"> catalog: Catalog name</span>
</span><span id="qualify_tables-43"><a href="#qualify_tables-43"><span class="linenos"> 43</span></a><span class="sd"> schema: A schema to populate</span>
</span><span id="qualify_tables-44"><a href="#qualify_tables-44"><span class="linenos"> 44</span></a><span class="sd"> infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs&#39; schemas.</span>
</span><span id="qualify_tables-45"><a href="#qualify_tables-45"><span class="linenos"> 45</span></a><span class="sd"> dialect: The dialect to parse catalog and schema into.</span>
</span><span id="qualify_tables-46"><a href="#qualify_tables-46"><span class="linenos"> 46</span></a>
</span><span id="qualify_tables-47"><a href="#qualify_tables-47"><span class="linenos"> 47</span></a><span class="sd"> Returns:</span>
</span><span id="qualify_tables-48"><a href="#qualify_tables-48"><span class="linenos"> 48</span></a><span class="sd"> The qualified expression.</span>
</span><span id="qualify_tables-49"><a href="#qualify_tables-49"><span class="linenos"> 49</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="qualify_tables-50"><a href="#qualify_tables-50"><span class="linenos"> 50</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">&quot;_q_&quot;</span><span class="p">)</span>
</span><span id="qualify_tables-51"><a href="#qualify_tables-51"><span class="linenos"> 51</span></a> <span class="n">db</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">db</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="qualify_tables-52"><a href="#qualify_tables-52"><span class="linenos"> 52</span></a> <span class="n">catalog</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">parse_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> <span class="k">if</span> <span class="n">catalog</span> <span class="k">else</span> <span class="kc">None</span>
</span><span id="qualify_tables-53"><a href="#qualify_tables-53"><span class="linenos"> 53</span></a>
</span><span id="qualify_tables-54"><a href="#qualify_tables-54"><span class="linenos"> 54</span></a> <span class="k">def</span> <span class="nf">_qualify</span><span class="p">(</span><span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
</span><span id="qualify_tables-55"><a href="#qualify_tables-55"><span class="linenos"> 55</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
</span><span id="qualify_tables-56"><a href="#qualify_tables-56"><span class="linenos"> 56</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="qualify_tables-57"><a href="#qualify_tables-57"><span class="linenos"> 57</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">,</span> <span class="n">db</span><span class="p">)</span>
</span><span id="qualify_tables-58"><a href="#qualify_tables-58"><span class="linenos"> 58</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">)</span> <span class="ow">and</span> <span class="n">table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;db&quot;</span><span class="p">):</span>
</span><span id="qualify_tables-59"><a href="#qualify_tables-59"><span class="linenos"> 59</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">,</span> <span class="n">catalog</span><span class="p">)</span>
</span><span id="qualify_tables-60"><a href="#qualify_tables-60"><span class="linenos"> 60</span></a>
</span><span id="qualify_tables-61"><a href="#qualify_tables-61"><span class="linenos"> 61</span></a> <span class="k">if</span> <span class="p">(</span><span class="n">db</span> <span class="ow">or</span> <span class="n">catalog</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">):</span>
</span><span id="qualify_tables-62"><a href="#qualify_tables-62"><span class="linenos"> 62</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Query</span><span class="p">)):</span>
</span><span id="qualify_tables-63"><a href="#qualify_tables-63"><span class="linenos"> 63</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="qualify_tables-64"><a href="#qualify_tables-64"><span class="linenos"> 64</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="qualify_tables-65"><a href="#qualify_tables-65"><span class="linenos"> 65</span></a>
</span><span id="qualify_tables-66"><a href="#qualify_tables-66"><span class="linenos"> 66</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
</span><span id="qualify_tables-67"><a href="#qualify_tables-67"><span class="linenos"> 67</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
</span><span id="qualify_tables-68"><a href="#qualify_tables-68"><span class="linenos"> 68</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
</span><span id="qualify_tables-69"><a href="#qualify_tables-69"><span class="linenos"> 69</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
</span><span id="qualify_tables-70"><a href="#qualify_tables-70"><span class="linenos"> 70</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="qualify_tables-71"><a href="#qualify_tables-71"><span class="linenos"> 71</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="qualify_tables-72"><a href="#qualify_tables-72"><span class="linenos"> 72</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">&quot;*&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
</span><span id="qualify_tables-73"><a href="#qualify_tables-73"><span class="linenos"> 73</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span>
</span><span id="qualify_tables-74"><a href="#qualify_tables-74"><span class="linenos"> 74</span></a>
</span><span id="qualify_tables-75"><a href="#qualify_tables-75"><span class="linenos"> 75</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">):</span>
</span><span id="qualify_tables-76"><a href="#qualify_tables-76"><span class="linenos"> 76</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
</span><span id="qualify_tables-77"><a href="#qualify_tables-77"><span class="linenos"> 77</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
</span><span id="qualify_tables-78"><a href="#qualify_tables-78"><span class="linenos"> 78</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
</span><span id="qualify_tables-79"><a href="#qualify_tables-79"><span class="linenos"> 79</span></a>
</span><span id="qualify_tables-80"><a href="#qualify_tables-80"><span class="linenos"> 80</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="qualify_tables-81"><a href="#qualify_tables-81"><span class="linenos"> 81</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="qualify_tables-82"><a href="#qualify_tables-82"><span class="linenos"> 82</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
</span><span id="qualify_tables-83"><a href="#qualify_tables-83"><span class="linenos"> 83</span></a>
</span><span id="qualify_tables-84"><a href="#qualify_tables-84"><span class="linenos"> 84</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="qualify_tables-85"><a href="#qualify_tables-85"><span class="linenos"> 85</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="qualify_tables-86"><a href="#qualify_tables-86"><span class="linenos"> 86</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="qualify_tables-87"><a href="#qualify_tables-87"><span class="linenos"> 87</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="qualify_tables-88"><a href="#qualify_tables-88"><span class="linenos"> 88</span></a> <span class="c1"># Don&#39;t add the pivot&#39;s alias to the pivoted table, use the table&#39;s name instead</span>
</span><span id="qualify_tables-89"><a href="#qualify_tables-89"><span class="linenos"> 89</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span> <span class="o">==</span> <span class="n">name</span><span class="p">:</span>
</span><span id="qualify_tables-90"><a href="#qualify_tables-90"><span class="linenos"> 90</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span>
</span><span id="qualify_tables-91"><a href="#qualify_tables-91"><span class="linenos"> 91</span></a>
</span><span id="qualify_tables-92"><a href="#qualify_tables-92"><span class="linenos"> 92</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span>
</span><span id="qualify_tables-93"><a href="#qualify_tables-93"><span class="linenos"> 93</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="qualify_tables-94"><a href="#qualify_tables-94"><span class="linenos"> 94</span></a>
</span><span id="qualify_tables-95"><a href="#qualify_tables-95"><span class="linenos"> 95</span></a> <span class="n">table_aliases</span><span class="p">[</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">parts</span><span class="p">)]</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span>
</span><span id="qualify_tables-96"><a href="#qualify_tables-96"><span class="linenos"> 96</span></a> <span class="n">source</span><span class="o">.</span><span class="n">alias</span>
</span><span id="qualify_tables-97"><a href="#qualify_tables-97"><span class="linenos"> 97</span></a> <span class="p">)</span>
</span><span id="qualify_tables-98"><a href="#qualify_tables-98"><span class="linenos"> 98</span></a>
</span><span id="qualify_tables-99"><a href="#qualify_tables-99"><span class="linenos"> 99</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
</span><span id="qualify_tables-84"><a href="#qualify_tables-84"><span class="linenos"> 84</span></a> <span class="n">table_aliases</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="qualify_tables-85"><a href="#qualify_tables-85"><span class="linenos"> 85</span></a>
</span><span id="qualify_tables-86"><a href="#qualify_tables-86"><span class="linenos"> 86</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="qualify_tables-87"><a href="#qualify_tables-87"><span class="linenos"> 87</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="qualify_tables-88"><a href="#qualify_tables-88"><span class="linenos"> 88</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;pivots&quot;</span><span class="p">)</span>
</span><span id="qualify_tables-89"><a href="#qualify_tables-89"><span class="linenos"> 89</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="qualify_tables-90"><a href="#qualify_tables-90"><span class="linenos"> 90</span></a> <span class="c1"># Don&#39;t add the pivot&#39;s alias to the pivoted table, use the table&#39;s name instead</span>
</span><span id="qualify_tables-91"><a href="#qualify_tables-91"><span class="linenos"> 91</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span> <span class="o">==</span> <span class="n">name</span><span class="p">:</span>
</span><span id="qualify_tables-92"><a href="#qualify_tables-92"><span class="linenos"> 92</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span>
</span><span id="qualify_tables-93"><a href="#qualify_tables-93"><span class="linenos"> 93</span></a>
</span><span id="qualify_tables-94"><a href="#qualify_tables-94"><span class="linenos"> 94</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span>
</span><span id="qualify_tables-95"><a href="#qualify_tables-95"><span class="linenos"> 95</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="qualify_tables-96"><a href="#qualify_tables-96"><span class="linenos"> 96</span></a>
</span><span id="qualify_tables-97"><a href="#qualify_tables-97"><span class="linenos"> 97</span></a> <span class="n">table_aliases</span><span class="p">[</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">parts</span><span class="p">)]</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span>
</span><span id="qualify_tables-98"><a href="#qualify_tables-98"><span class="linenos"> 98</span></a> <span class="n">source</span><span class="o">.</span><span class="n">alias</span>
</span><span id="qualify_tables-99"><a href="#qualify_tables-99"><span class="linenos"> 99</span></a> <span class="p">)</span>
</span><span id="qualify_tables-100"><a href="#qualify_tables-100"><span class="linenos">100</span></a>
</span><span id="qualify_tables-101"><a href="#qualify_tables-101"><span class="linenos">101</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="qualify_tables-102"><a href="#qualify_tables-102"><span class="linenos">102</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
</span><span id="qualify_tables-103"><a href="#qualify_tables-103"><span class="linenos">103</span></a> <span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="qualify_tables-104"><a href="#qualify_tables-104"><span class="linenos">104</span></a> <span class="p">)</span>
</span><span id="qualify_tables-105"><a href="#qualify_tables-105"><span class="linenos">105</span></a>
</span><span id="qualify_tables-106"><a href="#qualify_tables-106"><span class="linenos">106</span></a> <span class="k">if</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
</span><span id="qualify_tables-107"><a href="#qualify_tables-107"><span class="linenos">107</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
</span><span id="qualify_tables-108"><a href="#qualify_tables-108"><span class="linenos">108</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="qualify_tables-109"><a href="#qualify_tables-109"><span class="linenos">109</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="qualify_tables-110"><a href="#qualify_tables-110"><span class="linenos">110</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
</span><span id="qualify_tables-111"><a href="#qualify_tables-111"><span class="linenos">111</span></a> <span class="n">source</span><span class="p">,</span>
</span><span id="qualify_tables-112"><a href="#qualify_tables-112"><span class="linenos">112</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
</span><span id="qualify_tables-113"><a href="#qualify_tables-113"><span class="linenos">113</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="qualify_tables-114"><a href="#qualify_tables-114"><span class="linenos">114</span></a> <span class="p">)</span>
</span><span id="qualify_tables-115"><a href="#qualify_tables-115"><span class="linenos">115</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
</span><span id="qualify_tables-116"><a href="#qualify_tables-116"><span class="linenos">116</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="qualify_tables-117"><a href="#qualify_tables-117"><span class="linenos">117</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="qualify_tables-118"><a href="#qualify_tables-118"><span class="linenos">118</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
</span><span id="qualify_tables-119"><a href="#qualify_tables-119"><span class="linenos">119</span></a> <span class="p">)</span>
</span><span id="qualify_tables-120"><a href="#qualify_tables-120"><span class="linenos">120</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="qualify_tables-121"><a href="#qualify_tables-121"><span class="linenos">121</span></a>
</span><span id="qualify_tables-122"><a href="#qualify_tables-122"><span class="linenos">122</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
</span><span id="qualify_tables-123"><a href="#qualify_tables-123"><span class="linenos">123</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="qualify_tables-124"><a href="#qualify_tables-124"><span class="linenos">124</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="qualify_tables-125"><a href="#qualify_tables-125"><span class="linenos">125</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
</span><span id="qualify_tables-126"><a href="#qualify_tables-126"><span class="linenos">126</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">))</span>
</span><span id="qualify_tables-127"><a href="#qualify_tables-127"><span class="linenos">127</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="qualify_tables-128"><a href="#qualify_tables-128"><span class="linenos">128</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
</span><span id="qualify_tables-129"><a href="#qualify_tables-129"><span class="linenos">129</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="qualify_tables-130"><a href="#qualify_tables-130"><span class="linenos">130</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
</span><span id="qualify_tables-131"><a href="#qualify_tables-131"><span class="linenos">131</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span>
</span><span id="qualify_tables-132"><a href="#qualify_tables-132"><span class="linenos">132</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
</span><span id="qualify_tables-133"><a href="#qualify_tables-133"><span class="linenos">133</span></a> <span class="p">):</span>
</span><span id="qualify_tables-134"><a href="#qualify_tables-134"><span class="linenos">134</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span>
</span><span id="qualify_tables-135"><a href="#qualify_tables-135"><span class="linenos">135</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="qualify_tables-136"><a href="#qualify_tables-136"><span class="linenos">136</span></a>
</span><span id="qualify_tables-137"><a href="#qualify_tables-137"><span class="linenos">137</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="qualify_tables-138"><a href="#qualify_tables-138"><span class="linenos">138</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span>
</span><span id="qualify_tables-139"><a href="#qualify_tables-139"><span class="linenos">139</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
</span><span id="qualify_tables-140"><a href="#qualify_tables-140"><span class="linenos">140</span></a>
</span><span id="qualify_tables-141"><a href="#qualify_tables-141"><span class="linenos">141</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span>
</span><span id="qualify_tables-142"><a href="#qualify_tables-142"><span class="linenos">142</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
</span><span id="qualify_tables-143"><a href="#qualify_tables-143"><span class="linenos">143</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="qualify_tables-144"><a href="#qualify_tables-144"><span class="linenos">144</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;table&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="qualify_tables-145"><a href="#qualify_tables-145"><span class="linenos">145</span></a>
</span><span id="qualify_tables-146"><a href="#qualify_tables-146"><span class="linenos">146</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="qualify_tables-101"><a href="#qualify_tables-101"><span class="linenos">101</span></a> <span class="n">_qualify</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
</span><span id="qualify_tables-102"><a href="#qualify_tables-102"><span class="linenos">102</span></a>
</span><span id="qualify_tables-103"><a href="#qualify_tables-103"><span class="linenos">103</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
</span><span id="qualify_tables-104"><a href="#qualify_tables-104"><span class="linenos">104</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
</span><span id="qualify_tables-105"><a href="#qualify_tables-105"><span class="linenos">105</span></a> <span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="qualify_tables-106"><a href="#qualify_tables-106"><span class="linenos">106</span></a> <span class="p">)</span>
</span><span id="qualify_tables-107"><a href="#qualify_tables-107"><span class="linenos">107</span></a>
</span><span id="qualify_tables-108"><a href="#qualify_tables-108"><span class="linenos">108</span></a> <span class="k">if</span> <span class="n">infer_csv_schemas</span> <span class="ow">and</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
</span><span id="qualify_tables-109"><a href="#qualify_tables-109"><span class="linenos">109</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
</span><span id="qualify_tables-110"><a href="#qualify_tables-110"><span class="linenos">110</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="qualify_tables-111"><a href="#qualify_tables-111"><span class="linenos">111</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
</span><span id="qualify_tables-112"><a href="#qualify_tables-112"><span class="linenos">112</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
</span><span id="qualify_tables-113"><a href="#qualify_tables-113"><span class="linenos">113</span></a> <span class="n">source</span><span class="p">,</span>
</span><span id="qualify_tables-114"><a href="#qualify_tables-114"><span class="linenos">114</span></a> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)},</span>
</span><span id="qualify_tables-115"><a href="#qualify_tables-115"><span class="linenos">115</span></a> <span class="n">match_depth</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="qualify_tables-116"><a href="#qualify_tables-116"><span class="linenos">116</span></a> <span class="p">)</span>
</span><span id="qualify_tables-117"><a href="#qualify_tables-117"><span class="linenos">117</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
</span><span id="qualify_tables-118"><a href="#qualify_tables-118"><span class="linenos">118</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="qualify_tables-119"><a href="#qualify_tables-119"><span class="linenos">119</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
</span><span id="qualify_tables-120"><a href="#qualify_tables-120"><span class="linenos">120</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
</span><span id="qualify_tables-121"><a href="#qualify_tables-121"><span class="linenos">121</span></a> <span class="p">)</span>
</span><span id="qualify_tables-122"><a href="#qualify_tables-122"><span class="linenos">122</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;alias&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="qualify_tables-123"><a href="#qualify_tables-123"><span class="linenos">123</span></a>
</span><span id="qualify_tables-124"><a href="#qualify_tables-124"><span class="linenos">124</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
</span><span id="qualify_tables-125"><a href="#qualify_tables-125"><span class="linenos">125</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;this&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
</span><span id="qualify_tables-126"><a href="#qualify_tables-126"><span class="linenos">126</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="qualify_tables-127"><a href="#qualify_tables-127"><span class="linenos">127</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
</span><span id="qualify_tables-128"><a href="#qualify_tables-128"><span class="linenos">128</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">&quot;columns&quot;</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">))</span>
</span><span id="qualify_tables-129"><a href="#qualify_tables-129"><span class="linenos">129</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="qualify_tables-130"><a href="#qualify_tables-130"><span class="linenos">130</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
</span><span id="qualify_tables-131"><a href="#qualify_tables-131"><span class="linenos">131</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="qualify_tables-132"><a href="#qualify_tables-132"><span class="linenos">132</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
</span><span id="qualify_tables-133"><a href="#qualify_tables-133"><span class="linenos">133</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span>
</span><span id="qualify_tables-134"><a href="#qualify_tables-134"><span class="linenos">134</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
</span><span id="qualify_tables-135"><a href="#qualify_tables-135"><span class="linenos">135</span></a> <span class="p">):</span>
</span><span id="qualify_tables-136"><a href="#qualify_tables-136"><span class="linenos">136</span></a> <span class="c1"># Mutates the table by attaching an alias to it</span>
</span><span id="qualify_tables-137"><a href="#qualify_tables-137"><span class="linenos">137</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="qualify_tables-138"><a href="#qualify_tables-138"><span class="linenos">138</span></a>
</span><span id="qualify_tables-139"><a href="#qualify_tables-139"><span class="linenos">139</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
</span><span id="qualify_tables-140"><a href="#qualify_tables-140"><span class="linenos">140</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">db</span><span class="p">:</span>
</span><span id="qualify_tables-141"><a href="#qualify_tables-141"><span class="linenos">141</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">table_aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;.&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">p</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">column</span><span class="o">.</span><span class="n">parts</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
</span><span id="qualify_tables-142"><a href="#qualify_tables-142"><span class="linenos">142</span></a>
</span><span id="qualify_tables-143"><a href="#qualify_tables-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="n">table_alias</span><span class="p">:</span>
</span><span id="qualify_tables-144"><a href="#qualify_tables-144"><span class="linenos">144</span></a> <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">COLUMN_PARTS</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
</span><span id="qualify_tables-145"><a href="#qualify_tables-145"><span class="linenos">145</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="qualify_tables-146"><a href="#qualify_tables-146"><span class="linenos">146</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">&quot;table&quot;</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
</span><span id="qualify_tables-147"><a href="#qualify_tables-147"><span class="linenos">147</span></a>
</span><span id="qualify_tables-148"><a href="#qualify_tables-148"><span class="linenos">148</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>
@ -375,6 +379,7 @@
<li><strong>db:</strong> Database name</li>
<li><strong>catalog:</strong> Catalog name</li>
<li><strong>schema:</strong> A schema to populate</li>
<li><strong>infer_csv_schemas:</strong> Whether to scan READ_CSV calls in order to infer the CSVs' schemas.</li>
<li><strong>dialect:</strong> The dialect to parse catalog and schema into.</li>
</ul>

File diff suppressed because it is too large Load diff

View file

@ -3209,7 +3209,7 @@ prefix are statically known.</p>
<div class="attr variable">
<span class="name">DATETRUNC_COMPARISONS</span> =
<input id="DATETRUNC_COMPARISONS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#In">sqlglot.expressions.In</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;}</span>
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#In">sqlglot.expressions.In</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;}</span>
</div>
@ -3289,7 +3289,7 @@ prefix are statically known.</p>
<section id="JOINS">
<div class="attr variable">
<span class="name">JOINS</span> =
<span class="default_value">{(&#39;RIGHT&#39;, &#39;&#39;), (&#39;&#39;, &#39;INNER&#39;), (&#39;RIGHT&#39;, &#39;OUTER&#39;), (&#39;&#39;, &#39;&#39;)}</span>
<span class="default_value">{(&#39;RIGHT&#39;, &#39;OUTER&#39;), (&#39;&#39;, &#39;INNER&#39;), (&#39;RIGHT&#39;, &#39;&#39;), (&#39;&#39;, &#39;&#39;)}</span>
</div>

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because it is too large Load diff

515
posts/onboarding.md Normal file
View file

@ -0,0 +1,515 @@
# Table of Contents
- [Table of Contents](#table-of-contents)
- [Onboarding](#onboarding)
- [Tokenizer](#tokenizer)
- [Parser](#parser)
- [Token Index](#token-index)
- [Matching text](#matching-text)
- [Utility methods](#utility-methods)
- [Command fallback](#command-fallback)
- [Expression parsing](#expression-parsing)
- [Generator](#generator)
- [Generating queries](#generating-queries)
- [Utility methods](#utility-methods-1)
- [Pretty print](#pretty-print)
- [Schema](#schema)
- [Optimizer](#optimizer)
- [Optimization rules](#optimization-rules)
- [Qualify](#qualify)
- [Identifier normalization](#identifier-normalization)
- [Qualifying tables \& columns](#qualifying-tables--columns)
- [Type annotation](#type-annotation)
- [Dialects](#dialects)
- [Implementing a custom dialect](#implementing-a-custom-dialect)
- [Column Level Lineage](#column-level-lineage)
- [Implementation details](#implementation-details)
## Onboarding
This document aims to familiarize the reader with SQLGlot's codebase & architecture.
Generally, some background knowledge about programming languages / compilers is required. A good starting point is [Crafting Interpreters](https://craftinginterpreters.com/) by Robert Nystrom, which served as the foundation when SQLGlot was initially created.
At a high level, SQLGlot transpilation involves three modules:
- [Tokenizer](#tokenizer): Converts raw code into a sequence of “tokens”, one for each word/symbol
- [Parser](#parser): Converts sequence of tokens into an abstract syntax tree representing the semantics of the raw code
- [Generator](#generator): Converts an abstract syntax tree into SQL code
SQLGlot can transpile SQL between different _SQL dialects_, which are usually associated with different database systems.
Each dialect has unique syntax that is implemented by overriding the base versions of the three modules. A dialect definition may override any or all of the three base modules.
The base versions of the modules implement the `sqlglot` dialect (sometimes referred to as "base dialect"), which is designed to accommodate as many common syntax elements as possible across the other supported dialects, thus preventing code duplication.
SQLGlot includes other modules which are not required for basic transpilation, such as the [Optimizer](#optimizer) and Executor. The Optimizer modifies an abstract syntax tree for other uses (e.g., inferring column-level lineage). The Executor runs SQL code in Python, but its engine is still in an experimental stage, so some functionality may be missing.
The rest of this document describes the three base modules, dialect-specific overrides, and the Optimizer module.
## Tokenizer
The tokenizer module (`tokens.py`) is responsible for breaking down SQL code into tokens (like keywords, identifiers, etc.), which are the smallest units of meaningful information. This process is also known as lexing/lexical analysis.
Python is not designed for maximum processing speed/performance, so SQLGlot maintains an equivalent [Rust version of the tokenizer](https://tobikodata.com/sqlglot-jumps-on-the-rust-bandwagon.html) in `sqlglotrs/tokenizer.rs` for improved performance.
> [!IMPORTANT]
> Changes in the tokenization logic must be reflected in both the Python and the Rust tokenizer. The goal is for the two implementations to be similar so that there is less cognitive effort to port code between them.
This example demonstrates the results of using the Tokenizer to tokenize the SQL query `SELECT b FROM table WHERE c = 1`:
```python
from sqlglot import tokenize
tokens = tokenize("SELECT b FROM table WHERE c = 1")
for token in tokens:
print(token)
# <Token token_type: <TokenType.SELECT: 'SELECT'>, text: 'SELECT', line: 1, col: 6, start: 0, end: 5, comments: []>
# <Token token_type: <TokenType.VAR: 'VAR'>, text: 'b', line: 1, col: 8, start: 7, end: 7, comments: []>
# <Token token_type: <TokenType.FROM: 'FROM'>, text: 'FROM', line: 1, col: 13, start: 9, end: 12, comments: []>
# <Token token_type: <TokenType.TABLE: 'TABLE'>, text: 'table', line: 1, col: 19, start: 14, end: 18, comments: []>
# <Token token_type: <TokenType.WHERE: 'WHERE'>, text: 'WHERE', line: 1, col: 25, start: 20, end: 24, comments: []>
# <Token token_type: <TokenType.VAR: 'VAR'>, text: 'c', line: 1, col: 27, start: 26, end: 26, comments: []>
# <Token token_type: <TokenType.EQ: 'EQ'>, text: '=', line: 1, col: 29, start: 28, end: 28, comments: []>
# <Token token_type: <TokenType.NUMBER: 'NUMBER'>, text: '1', line: 1, col: 31, start: 30, end: 30, comments: []>
```
The Tokenizer scans the query and converts groups of symbols (or _lexemes_), such as words (e.g., `b`) and operators (e.g., `=`), into tokens. For example, the `VAR` token is used to represent the identifier `b`, whereas the `EQ` token is used to represent the "equals" operator.
Each token contains information such as its type (`token_type`), the lexeme (`text`) it encapsulates and other metadata such as the lexeme's line (`line`) and column (`col`), which are used to accurately report errors.
SQLGlots `TokenType` enum provides an indirection layer between lexemes and their types. For example, `!=` and `<>` are often used interchangeably to represent "not equals", so SQLGlot groups them together by mapping them both to `TokenType.NEQ`.
The `Tokenizer.KEYWORDS` and `Tokenizer.SINGLE_TOKENS` are two important dictionaries that map lexemes to the corresponding `TokenType` enum values.
## Parser
The parser module takes the list of tokens generated by the tokenizer and builds an [abstract syntax tree (AST)](https://en.wikipedia.org/wiki/Abstract_syntax_tree) from them.
Generally, an AST stores the _meaningful_ constituent components of a SQL statement. For example, to represent a `SELECT` query we can only store its projections, filters, aggregation expressions and so on, but the `SELECT`, `WHERE` and `GROUP BY` keywords don't need to be preserved, since they are implied.
In SQLGlot's case, the AST usually stores information that captures the _semantics_ of an expression, i.e. its meaning, which is what allows it to transpile SQL code between different dialects. A blog post that explains this idea at a high-level can [be found here](https://tobikodata.com/transpiling_sql1.html).
The parser processes the tokens corresponding to a SQL statement sequentially and produces an AST to represent it. For example, if a statements first token is `CREATE`, the parser will determine what is being created by examining the next token (which could be `SCHEMA`, `TABLE`, `VIEW`, or something else).
Each semantic concept is represented by an _expression_ in the AST. Converting tokens to expressions is the primary task of the parser.
As the AST is a core concept of SQLGlot, a more detailed tutorial on its representation, traversal, and transformation can [be found here](https://github.com/tobymao/sqlglot/blob/main/posts/ast_primer.md).
The next sections describe how the parser processes a SQL statements list of tokens, followed by an explanation of how dialect-specific parsing works.
### Token Index
The parser processes a token list sequentially, and maintains an index/cursor that points to the next token to be consumed.
Once the current token has been successfully processed, the parser moves to the next token in the sequence by calling `_advance()` to increment the index.
If a token has been consumed but could not be parsed, the parser can move backward to the previous token by calling `_retreat()` to decrement the index.
In some scenarios, the parsers behavior depends on both the current and surrounding tokens. The parser can “peek” at the previous and next tokens without consuming them by accessing the `_prev` and `_next` properties, respectively.
### Matching tokens and text
When parsing a SQL statement, the parser must identify specific sets of keywords to correctly build the AST. It does so by “matching” sets of tokens or keywords to infer the structure of the statement.
For example, these matches occur when parsing the window specification `SUM(x) OVER (PARTITION BY y)`:
1. The `SUM` and `(` tokens are matched and `x` is parsed as an identifier
2. The `)` token is matched
2. The `OVER` and `(` tokens are matched
2. The `PARTITION` and `BY` tokens are matched
3. The partition clause `y` is parsed
4. The `)` token is matched
The family of `_match` methods perform different varieties of matching. They return `True` and advance the index if there is a match, or return `None` if the tokens do not match.
The most commonly used `_match` methods are:
Method | Use
-------------------------|-----
**`_match(type)`** | Attempt to match a single token with a specific `TokenType`
**`_match_set(types)`** | Attempt to match a single token in a set of `TokenType`s
**`_match_text_seq(texts)`** | Attempt to match a continuous sequence of strings/texts
**`_match_texts(texts)`** | Attempt to match a keyword in a set of strings/texts
### `_parse` methods
SQLGlots parser follows the “recursive descent” approach, meaning that it relies on mutually recursive methods in order to understand SQL syntax and the various precedence levels between combined SQL expressions.
For example, the SQL statement `CREATE TABLE a (col1 INT)` will be parsed into an `exp.Create` expression that includes the kind of object being created (table) and its schema (table name, column names and types). The schema will be parsed into an `exp.Schema` node that contains one column definition `exp.ColumnDef` node for each column.
The relationships between SQL constructs are encoded in methods whose names begin with “_parse_”, such as `_parse_create()`. We refer to these as “parsing methods”.
For instance, to parse the statement above the entrypoint method `_parse_create()` is called. Then:
- In `_parse_create()`, the parser determines that a `TABLE` is being created
- Because a table is being created, the table name is expected next and thus `_parse_table_parts()` is invoked; This is because the table name may have multiple parts such as `catalog.schema.table`
- Having processed the table name, the parser continues by parsing the schema definition using `_parse_schema()`
A key aspect of the SQLGlot parser is that the parsing methods are composable and can be passed as arguments. We describe how that works in the next section.
### Utility methods
SQLGlot provides helper methods for common parsing tasks. They should be used whenever possible to reduce code duplication and manual token/text matching.
Many helper methods take a parsing method argument, which is invoked to parse part(s) of the clause they're supposed to parse.
For example, these helper methods parse collections of SQL objects like `col1, col2`, `(PARTITION BY y)`, and `(colA TEXT, colB INT)`, respectively:
Method | Use
-------------------------|-----
**`_parse_csv(parse_method)`** | Attempt to parse comma-separated values using the appropriate `parse_method` callable
**`_parse_wrapped(parse_method)`** | Attempt to parse a specific construct that is (optionally) wrapped in parentheses
**`_parse_wrapped_csv(parse_method)`** | Attempt to parse comma-separated values that are (optionally) wrapped in parentheses
### Command fallback
The SQL language specification and dialect variations are vast, and SQLGlot cannot parse every possible statement.
Instead of erroring on statements it cannot parse, SQLGlot falls back to storing the code that cannot be parsed in an `exp.Command` expression.
This allows SQLGlot to return the code unmodified even though it cannot parse it. Because the code is unmodified, any dialect-specific components will not be transpiled.
### Dialect-specific parsing
The base parsers goal is to represent as many common constructs from different SQL dialects as possible. This makes the parser more lenient and less-repetitive/concise.
> [!WARNING]
> SQLGlot does not aim to be a SQL validator, so it may not detect certain syntax errors.
Dialect-specific parser behavior is implemented in two ways: feature flags and parser overrides.
If two different parsing behaviors are common across dialects, the base parser may implement both and use feature flags to determine which should be used for a specific dialect. In contrast, parser overrides directly replace specific base parser methods.
Therefore, each dialects Parser class may specify:
- Feature flags such as `SUPPORTS_IMPLICIT_UNNEST` or `SUPPORTS_PARTITION_SELECTION`, which are used to control the behavior of base parser methods.
- Sets of tokens that play a similar role, such as `RESERVED_TOKENS` that cannot be used as object names.
- Sets of `token -> Callable` mappings, implemented with Python lambda functions
- When the parser comes across the token (string or `TokenType`) on the left-hand side it invokes the mapping function on the right to create the corresponding Expression / AST node.
- The lambda function may return an expression directly or a `_parse_ method` that determines what expression should be returned.
- The mappings are grouped by general semantic type (e.g., functions, parsers for `ALTER` statements, etc.), with each type having its own dictionary.
An example `token -> Callable` mapping is the `FUNCTIONS` dictionary that builds the appropriate `exp.Func` node based on the string key:
```Python3
FUNCTIONS: t.Dict[str, t.Callable] = {
"LOG2": lambda args: exp.Log(this=exp.Literal.number(2), expression=seq_get(args, 0)),
"LOG10": lambda args: exp.Log(this=exp.Literal.number(10), expression=seq_get(args, 0)),
"MOD": build_mod,
…,
}
```
In this dialect, the `LOG2()` function calculates a logarithm of base 2, which is represented in SQLGlot by the general `exp.Log` expression.
The logarithm base and the user-specified value to `log` are stored in the node's arguments. The former is stored as an integer literal in the `this` argument, and the latter is stored in the `expression` argument.
## Generator
After the parser has created an AST, the generator module is responsible for converting it back into SQL code.
Each dialects Generator class may specify:
- A set of flags such as `ALTER_TABLE_INCLUDE_COLUMN_KEYWORD`. As with the parser, these flags control the behavior of base Generator methods.
- Set of `Expression -> str` mappings
- The generator traverses the AST recursively and produces the equivalent string for each expression it visits. This can be thought of as the reverse operation of the parser, where expressions / AST nodes are converted back to strings.
- The mappings are grouped by general semantic type (e.g., data type expressions), with each type having its own dictionary.
The `Expression -> str` mappings can be implemented as either:
- Entries in the `TRANSFORMS` dictionary, which are best suited for single-line generations.
- Functions with names of the form `<expr_name>_sql(...)`. For example, to generate SQL for an `exp.SessionParameter` node we can override the base generator method by defining the method `def sessionparameter_sql(...)` in a dialects Generator class.
### Generating queries
The key method in the Generator module is the `sql()` function, which is responsible for generating the string representation of each expression.
First, it locates the mapping of expression to string or generator `Callable` by examining the keys of the `TRANSFORMS` dictionary and searching the `Generator`'s methods for an `<exprname>_sql()` method.
It then calls the appropriate callable to produce the corresponding SQL code.
> [!IMPORTANT]
> If there is both a `TRANSFORM` entry and an `<expr_name>_sql(...)` method for a given expression, the generator will use the former to convert the expression into a string.
### Utility methods
The Generator defines helper methods containing quality-of-life abstractions over the `sql()` method, such as:
Method | Use
-------------------------|-----
**`expressions()`** | Generates the string representation for a list of `Expression`s, employing a series of arguments to help with their formatting
**`func()`, `rename_func()`** | Given a function name and an `Expression`, returns the string representation of a function call by generating the expression's args as the func args.
These methods should be used whenever possible to reduce code duplication.
### Pretty print
Pretty printing refers to the process of generating formatted and consistently styled SQL, which improves readability, debugging, and code reviewing.
SQLGlot generates an ASTs SQL code on a single line by default, but users may specify that it be `pretty` and include new lines and indenting.
It is up to the developer to ensure that whitespaces and newlines are embedded in the SQL correctly. To aid in that process, the Generator class provides the `sep()` and `seg()` helper methods.
## Dialects
We briefly mentioned dialect-specific behaviors while describing the SQLGlot base Tokenizer, Parser, and Generator modules. This section expands on how to specify a new SQL dialect.
As [described above](#dialect-specific-parsing), SQLGlot attempts to bridge dialect-specific variations in a "superset" dialect, which we refer to as the _sqlglot_ dialect.
All other SQL dialects have their own Dialect subclass whose Tokenizer, Parser and Generator components extend or override the base modules as needed.
The base dialect components are defined in `tokens.py`, `parser.py` and `generator.py`. Dialect definitions can be found under `dialects/<dialect>.py`.
When adding features that will be used by multiple dialects, it's best to place the common/overlapping parts in the base _sqlglot_ dialect to prevent code repetition across other dialects. Any dialect-specific features that cannot (or should not) be repeated can be specified in the dialects subclass.
The Dialect class contains flags that are visible to its Tokenizer, Parser and Generator components. Flags are only added in a Dialect when they need to be visible to at least two components, in order to avoid code duplication.
### Implementing a custom dialect
Creating a new SQL dialect may seem complicated at first, but it is actually quite simple in SQLGlot.
This example demonstrates defining a new dialect named `Custom` that extends or overrides components of the base Dialect modules:
```Python
from sqlglot import exp
from sqlglot.dialects.dialect import Dialect
from sqlglot.generator import Generator
from sqlglot.tokens import Tokenizer, TokenType
class Custom(Dialect):
class Tokenizer(Tokenizer):
QUOTES = ["'", '"'] # Strings can be delimited by either single or double quotes
IDENTIFIERS = ["`"] # Identifiers can be delimited by backticks
# Associates certain meaningful words with tokens that capture their intent
KEYWORDS = {
**Tokenizer.KEYWORDS,
"INT64": TokenType.BIGINT,
"FLOAT64": TokenType.DOUBLE,
}
class Parser(Parser):
# Specifies how certain tokens are mapped to function AST nodes
FUNCTIONS = {
**parser.Parser.FUNCTIONS,
"APPROX_PERCENTILE": exp.ApproxQuantile.from_arg_list,
}
# Specifies how a specific construct e.g. CONSTRAINT is parsed
def _parse_constraint(self) -> t.Optional[exp.Expression]:
return super()._parse_constraint() or self._parse_projection_def()
class Generator(Generator):
# Specifies how AST nodes, i.e. subclasses of exp.Expression, should be converted into SQL
TRANSFORMS = {
exp.Array: lambda self, e: f"[{self.expressions(e)}]",
}
# Specifies how AST nodes representing data types should be converted into SQL
TYPE_MAPPING = {
exp.DataType.Type.TINYINT: "INT64",
exp.DataType.Type.SMALLINT: "INT64",
...
}
```
Even though this example is a fairly realistic starting point, we strongly encourage you to study existing dialect implementations to understand how their various components can be modified.
## Schema
Previous sections described the SQLGlot components used for transpilation. This section and the next describe components needed to implement other SQLGlot functionality like column-level lineage.
A Schema represents the structure of a database schema, including the tables/views it contains and their column names and data types.
The file `schema.py` defines the abstract classes `Schema` and `AbstractMappingSchema`; the implementing class is `MappingSchema`.
A `MappingSchema` object is defined with a nested dictionary, as in this example:
```Python
schema = MappingSchema({"t": {"x": "int", "y": "datetime"}})
```
The keys of the top-level dictionary are table names (`t`), the keys of `t`s dictionary are its column names (`x` and `y`), and the values are the column data types (`int` and `datetime`).
A schema provides information required by other SQLGlot modules (most importantly, the optimizer and column level lineage).
Schema information is used to enrich ASTs with actions like replacing stars (`SELECT *`) with the names of the columns being selected from the upstream tables.
## Optimizer
The optimizer module in SQLGlot (`optimizer.py`) is responsible for producing canonicalized and efficient SQL queries by applying a series of optimization rules.
Optimizations include simplifying expressions, removing redundant operations, and rewriting queries for better performance.
The optimizer operates on the abstract syntax tree (AST) returned by the parser, transforming it into a more compact form while preserving the semantics of the original query.
> [!NOTE]
> The optimizer performs only logical optimization; The underlying engine is almost always better at optimizing for performance.
### Optimization rules
The optimizer essentially applies [a list of rules](https://sqlglot.com/sqlglot/optimizer.html) in a sequential manner, where each rule takes in an AST and produces a canonicalized and optimized version of it.
> [!WARNING]
> The rule order is important, as some rules depend on others to work properly. We discourage manually applying individual rules, which may result in erroneous or unpredictable behavior.
The first, foundational optimization task is to _standardize_ an AST, which is required by other optimization steps. These rules implement canonicalization:
#### Qualify
The most important rule in the optimizer is `qualify`, which is responsible for rewriting the AST such that all tables and columns are _normalized_ and _qualified_.
##### Identifier normalization
The normalization step (`normalize_identifiers.py`) transforms some identifiers by converting them into lower or upper case, ensuring the semantics are preserved in each case (e.g. by respecting case-sensitivity).
This transformation reflects how identifiers would be resolved by the engine corresponding to each SQL dialect, and plays a very important role in the standardization of the AST.
> [!NOTE]
> Some dialects (e.g. DuckDB) treat identifiers as case-insensitive even when they're quoted, so all identifiers are normalized.
Different dialects may have different normalization strategies. For instance, in Snowflake unquoted identifiers are normalized to uppercase.
This example demonstrates how the identifier `Foo` is normalized to lowercase `foo` in the default sqlglot dialect and uppercase `FOO` in the Snowflake dialect:
```Python
import sqlglot
from sqlglot.optimizer.normalize_identifiers import normalize_identifiers
normalize_identifiers("Foo").sql()
# 'foo'
normalize_identifiers("Foo", dialect="snowflake").sql("snowflake")
# 'FOO'
```
##### Qualifying tables and columns
After normalizing the identifiers, all table and column names are qualified so there is no ambiguity about the data sources referenced by the query.
This means that all table references are assigned an alias and all column names are prefixed with their source tables name.
This example demonstrates qualifying the query `SELECT col FROM tbl`, where `tbl` contains a single column `col`. Note that the tables schema is passed as an argument to `qualify()`:
```Python
import sqlglot
from sqlglot.optimizer.qualify import qualify
schema = {"tbl": {"col": "INT"}}
expression = sqlglot.parse_one("SELECT col FROM tbl")
qualify(expression, schema=schema).sql()
# 'SELECT "tbl"."col" AS "col" FROM "tbl" AS "tbl"'
```
The `qualify` rule also offers a set of sub-rules that further canonicalize the query.
For instance, `qualify()` can expand stars such that each `SELECT *` is replaced by a projection list of the selected columns.
This example modifies the previous examples query to use `SELECT *`. `qualify()` expands the star and returns the same output as before:
```Python
import sqlglot
from sqlglot.optimizer.qualify import qualify
schema = {"tbl": {"col": "INT"}}
expression = sqlglot.parse_one("SELECT * FROM tbl")
qualify(expression, schema=schema).sql()
# 'SELECT "tbl"."col" AS "col" FROM "tbl" AS "tbl"'
```
#### Type Inference
Some SQL operators behavior changes based on the data type of its inputs.
For example, in some SQL dialects `+` can be used for either adding numbers or concatenating strings. The database uses its knowledge of column data types to determine which operation should be executed in a specific situation.
Like the database, SQLGlot needs to know column data types to perform some optimizations. In many cases, both transpilation and optimization need type information to work properly.
Type annotation begins with user-provided information about column data types for at least one table. SQLGlot then traverses the AST, propagating that type information and attempting to infer the type returned by each AST expression.
Users may need to provide column type information for multiple tables to successfully annotate all expressions in the AST.
## Column Level Lineage
Column-level lineage (CLL) traces the flow of data from column to column as tables select and modify columns from one another in a SQL codebase.
CLL provides critical information about data lineage that helps in understanding how data is derived, transformed, and used across different parts of a data pipeline or database system.
SQLGlots CLL implementation is defined in `lineage.py`. It operates on a collection of queries that `SELECT` from one another. It assumes that the graph/network of queries forms a directed acyclic graph (DAG) where no two tables `SELECT` from each other.
The user must provide:
A “target” query whose upstream column lineage should be traced
The queries linking all tables upstream of the target query
The schemas (column names and types) of the root tables in the DAG
In this example, we trace the lineage of the column `traced_col`:
```Python
from sqlglot.lineage import lineage
target_query = “””
WITH cte AS (
SELECT
traced_col
FROM
intermediate_table
)
SELECT
traced_col
FROM
cte
“””
node = lineage(
column="traced_col",
sql=target_query,
sources={"intermediate_table": "SELECT * FROM root_table"},
schema={"root_table": {"traced_col": "int"}},
)
```
The `lineage()` functions `sql` argument takes the target query, the `sources` argument takes a dictionary of source table names and the query that produces each table, and the `schema` argument takes the column names/types of the graphs root tables.
### Implementation details
This section describes how SQLGlot traces the lineage in the previous example.
Before lineage can be traced, the following preparatory steps must be carried out:
1. Replace the `sql` querys table references with their `sources` queries, such that we have a single standalone query.
This example demonstrates how the earlier examples reference to `intermediate_table` inside the CTE is replaced with the query that generated it, `SELECT * FROM root_table`, and given the alias `intermediate_table`:
```SQL
WITH cte AS (
SELECT
traced_col
FROM (
SELECT
*
FROM
root_table
) AS intermediate_table
)
SELECT
traced_col
FROM
cte
```
2. Qualify the query produced by the earlier step, expanding `*`s and qualifying all column references with the corresponding source tables:
```SQL
WITH cte AS (
SELECT
intermediate_table.traced_col
FROM (
SELECT
root_table.traced_col
FROM
root_table AS root_table
) AS intermediate_table
)
SELECT
cte.traced_col
FROM
cte AS cte
```
After these operations, the query is canonicalized and the lineage can be traced.
Tracing is a top-down process, meaning that the search starts from `cte.traced_col` (outer scope) and traverse inwards to find that its origin is `intermediate_table.traced_col`, which in turn is based on `root_table.traced_col`.
The search continues until the innermost scope has been visited. At that point all `sources` are exhausted, so the `schema` is searched to validate that the root table (`root_table` in our example) exists and that the target column `traced_col` is defined in it.
At each step, the column of interest (i.e., `cte.traced_col`, `intermediate_table.traced_col`. etc) is wrapped in a lineage object `Node` that is linked to its upstream nodes.
This approach incrementally builds a linked list of `Nodes` that traces the lineage for the target column: `root_table.traced_col -> intermediate_table.traced_col -> cte.traced_col`.
The `lineage()` output can be visualized using the `node.to_html()` function, which generates an HTML lineage graph using `vis.js`.
![Lineage](onboarding_images/lineage_img.png)

Binary file not shown.

After

Width:  |  Height:  |  Size: 188 KiB

View file

@ -34,10 +34,12 @@ setup(
"pandas",
"pandas-stubs",
"python-dateutil",
"pytz",
"pdoc",
"pre-commit",
"ruff==0.4.3",
"types-python-dateutil",
"types-pytz",
"typing_extensions",
"maturin>=1.4,<2.0",
],

View file

@ -6,6 +6,12 @@ from sqlglot.tokens import TokenType
class Athena(Trino):
class Tokenizer(Trino.Tokenizer):
KEYWORDS = {
**Trino.Tokenizer.KEYWORDS,
"UNLOAD": TokenType.COMMAND,
}
class Parser(Trino.Parser):
STATEMENT_PARSERS = {
**Trino.Parser.STATEMENT_PARSERS,

View file

@ -252,6 +252,7 @@ class BigQuery(Dialect):
TIME_MAPPING = {
"%D": "%m/%d/%y",
"%E6S": "%S.%f",
"%e": "%-d",
}
FORMAT_MAPPING = {
@ -401,6 +402,9 @@ class BigQuery(Dialect):
),
"TIMESTAMP_SECONDS": lambda args: exp.UnixToTime(this=seq_get(args, 0)),
"TO_JSON_STRING": exp.JSONFormat.from_arg_list,
"FORMAT_DATETIME": lambda args: exp.TimeToStr(
this=exp.TsOrDsToTimestamp(this=seq_get(args, 1)), format=seq_get(args, 0)
),
}
FUNCTION_PARSERS = {
@ -500,7 +504,7 @@ class BigQuery(Dialect):
table.set("db", exp.Identifier(this=parts[0]))
table.set("this", exp.Identifier(this=parts[1]))
if any("." in p.name for p in table.parts):
if isinstance(table.this, exp.Identifier) and any("." in p.name for p in table.parts):
catalog, db, this, *rest = (
exp.to_identifier(p, quoted=True)
for p in split_num_words(".".join(p.name for p in table.parts), ".", 3)
@ -583,6 +587,28 @@ class BigQuery(Dialect):
return bracket
def _parse_unnest(self, with_alias: bool = True) -> t.Optional[exp.Unnest]:
unnest = super()._parse_unnest(with_alias=with_alias)
if not unnest:
return None
unnest_expr = seq_get(unnest.expressions, 0)
if unnest_expr:
from sqlglot.optimizer.annotate_types import annotate_types
unnest_expr = annotate_types(unnest_expr)
# Unnesting a nested array (i.e array of structs) explodes the top-level struct fields,
# in contrast to other dialects such as DuckDB which flattens only the array by default
if unnest_expr.is_type(exp.DataType.Type.ARRAY) and any(
array_elem.is_type(exp.DataType.Type.STRUCT)
for array_elem in unnest_expr._type.expressions
):
unnest.set("explode_array", True)
return unnest
class Generator(generator.Generator):
EXPLICIT_SET_OP = True
INTERVAL_ALLOWS_PLURAL_FORM = False
@ -606,6 +632,7 @@ class BigQuery(Dialect):
NAMED_PLACEHOLDER_TOKEN = "@"
HEX_FUNC = "TO_HEX"
WITH_PROPERTIES_PREFIX = "OPTIONS"
SUPPORTS_EXPLODING_PROJECTIONS = False
TRANSFORMS = {
**generator.Generator.TRANSFORMS,
@ -878,8 +905,16 @@ class BigQuery(Dialect):
return super().table_parts(expression)
def timetostr_sql(self, expression: exp.TimeToStr) -> str:
this = expression.this if isinstance(expression.this, exp.TsOrDsToDate) else expression
return self.func("FORMAT_DATE", self.format_time(expression), this.this)
if isinstance(expression.this, exp.TsOrDsToTimestamp):
func_name = "FORMAT_DATETIME"
else:
func_name = "FORMAT_DATE"
this = (
expression.this
if isinstance(expression.this, (exp.TsOrDsToTimestamp, exp.TsOrDsToDate))
else expression
)
return self.func(func_name, self.format_time(expression), this.this)
def eq_sql(self, expression: exp.EQ) -> str:
# Operands of = cannot be NULL in BigQuery

View file

@ -1,10 +1,12 @@
from __future__ import annotations
import typing as t
import datetime
from sqlglot import exp, generator, parser, tokens
from sqlglot.dialects.dialect import (
Dialect,
NormalizationStrategy,
arg_max_or_min_no_count,
build_date_delta,
build_formatted_time,
@ -31,7 +33,7 @@ def _build_date_format(args: t.List) -> exp.TimeToStr:
timezone = seq_get(args, 2)
if timezone:
expr.set("timezone", timezone)
expr.set("zone", timezone)
return expr
@ -104,6 +106,28 @@ def _datetime_delta_sql(name: str) -> t.Callable[[Generator, DATEΤΙΜΕ_DELTA]
return _delta_sql
def _timestrtotime_sql(self: ClickHouse.Generator, expression: exp.TimeStrToTime):
tz = expression.args.get("zone")
datatype = exp.DataType.build(exp.DataType.Type.TIMESTAMP)
ts = expression.this
if tz:
# build a datatype that encodes the timezone as a type parameter, eg DateTime('America/Los_Angeles')
datatype = exp.DataType.build(
exp.DataType.Type.TIMESTAMPTZ, # Type.TIMESTAMPTZ maps to DateTime
expressions=[exp.DataTypeParam(this=tz)],
)
if isinstance(ts, exp.Literal):
# strip the timezone out of the literal, eg turn '2020-01-01 12:13:14-08:00' into '2020-01-01 12:13:14'
# this is because Clickhouse encodes the timezone as a data type parameter and throws an error if it's part of the timestamp string
ts_without_tz = (
datetime.datetime.fromisoformat(ts.name).replace(tzinfo=None).isoformat(sep=" ")
)
ts = exp.Literal.string(ts_without_tz)
return self.sql(exp.cast(ts, datatype, dialect=self.dialect))
class ClickHouse(Dialect):
NORMALIZE_FUNCTIONS: bool | str = False
NULL_ORDERING = "nulls_are_last"
@ -112,10 +136,15 @@ class ClickHouse(Dialect):
LOG_BASE_FIRST: t.Optional[bool] = None
FORCE_EARLY_ALIAS_REF_EXPANSION = True
# https://github.com/ClickHouse/ClickHouse/issues/33935#issue-1112165779
NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_SENSITIVE
UNESCAPED_SEQUENCES = {
"\\0": "\0",
}
CREATABLE_KIND_MAPPING = {"DATABASE": "SCHEMA"}
class Tokenizer(tokens.Tokenizer):
COMMENTS = ["--", "#", "#!", ("/*", "*/")]
IDENTIFIERS = ['"', "`"]
@ -424,6 +453,27 @@ class ClickHouse(Dialect):
"INDEX",
}
PLACEHOLDER_PARSERS = {
**parser.Parser.PLACEHOLDER_PARSERS,
TokenType.L_BRACE: lambda self: self._parse_query_parameter(),
}
def _parse_types(
self, check_func: bool = False, schema: bool = False, allow_identifiers: bool = True
) -> t.Optional[exp.Expression]:
dtype = super()._parse_types(
check_func=check_func, schema=schema, allow_identifiers=allow_identifiers
)
if isinstance(dtype, exp.DataType):
# Mark every type as non-nullable which is ClickHouse's default. This marker
# helps us transpile types from other dialects to ClickHouse, so that we can
# e.g. produce `CAST(x AS Nullable(String))` from `CAST(x AS TEXT)`. If there
# is a `NULL` value in `x`, the former would fail in ClickHouse without the
# `Nullable` type constructor
dtype.set("nullable", False)
return dtype
def _parse_extract(self) -> exp.Extract | exp.Anonymous:
index = self._index
this = self._parse_bitwise()
@ -433,7 +483,7 @@ class ClickHouse(Dialect):
# We return Anonymous here because extract and regexpExtract have different semantics,
# so parsing extract(foo, bar) into RegexpExtract can potentially break queries. E.g.,
# `extract('foobar', 'b')` works, but CH crashes for `regexpExtract('foobar', 'b')`.
# `extract('foobar', 'b')` works, but ClickHouse crashes for `regexpExtract('foobar', 'b')`.
#
# TODO: can we somehow convert the former into an equivalent `regexpExtract` call?
self._match(TokenType.COMMA)
@ -454,14 +504,11 @@ class ClickHouse(Dialect):
return this
def _parse_placeholder(self) -> t.Optional[exp.Expression]:
def _parse_query_parameter(self) -> t.Optional[exp.Expression]:
"""
Parse a placeholder expression like SELECT {abc: UInt32} or FROM {table: Identifier}
https://clickhouse.com/docs/en/sql-reference/syntax#defining-and-using-query-parameters
"""
if not self._match(TokenType.L_BRACE):
return None
this = self._parse_id_var()
self._match(TokenType.COLON)
kind = self._parse_types(check_func=False, allow_identifiers=False) or (
@ -589,7 +636,7 @@ class ClickHouse(Dialect):
if isinstance(expr, exp.Window):
# The window's func was parsed as Anonymous in base parser, fix its
# type to be CH style CombinedAnonymousAggFunc / AnonymousAggFunc
# type to be ClickHouse style CombinedAnonymousAggFunc / AnonymousAggFunc
expr.set("this", func)
elif params:
# Params have blocked super()._parse_function() from parsing the following window
@ -715,6 +762,7 @@ class ClickHouse(Dialect):
GROUPINGS_SEP = ""
SET_OP_MODIFIERS = False
SUPPORTS_TABLE_ALIAS_COLUMNS = False
VALUES_AS_TABLE = False
STRING_TYPE_MAPPING = {
exp.DataType.Type.CHAR: "String",
@ -741,7 +789,10 @@ class ClickHouse(Dialect):
exp.DataType.Type.ARRAY: "Array",
exp.DataType.Type.BIGINT: "Int64",
exp.DataType.Type.DATE32: "Date32",
exp.DataType.Type.DATETIME: "DateTime",
exp.DataType.Type.DATETIME64: "DateTime64",
exp.DataType.Type.TIMESTAMP: "DateTime",
exp.DataType.Type.TIMESTAMPTZ: "DateTime",
exp.DataType.Type.DOUBLE: "Float64",
exp.DataType.Type.ENUM: "Enum",
exp.DataType.Type.ENUM8: "Enum8",
@ -790,6 +841,7 @@ class ClickHouse(Dialect):
exp.CurrentDate: lambda self, e: self.func("CURRENT_DATE"),
exp.DateAdd: _datetime_delta_sql("DATE_ADD"),
exp.DateDiff: _datetime_delta_sql("DATE_DIFF"),
exp.DateStrToDate: rename_func("toDate"),
exp.DateSub: _datetime_delta_sql("DATE_SUB"),
exp.Explode: rename_func("arrayJoin"),
exp.Final: lambda self, e: f"{self.sql(e, 'this')} FINAL",
@ -810,8 +862,9 @@ class ClickHouse(Dialect):
"position", e.this, e.args.get("substr"), e.args.get("position")
),
exp.TimeToStr: lambda self, e: self.func(
"DATE_FORMAT", e.this, self.format_time(e), e.args.get("timezone")
"DATE_FORMAT", e.this, self.format_time(e), e.args.get("zone")
),
exp.TimeStrToTime: _timestrtotime_sql,
exp.TimestampAdd: _datetime_delta_sql("TIMESTAMP_ADD"),
exp.TimestampSub: _datetime_delta_sql("TIMESTAMP_SUB"),
exp.VarMap: lambda self, e: _lower_func(var_map_sql(self, e)),
@ -823,6 +876,7 @@ class ClickHouse(Dialect):
exp.UnixToTime: _unix_to_time_sql,
exp.TimestampTrunc: timestamptrunc_sql(zone=True),
exp.Variance: rename_func("varSamp"),
exp.SchemaCommentProperty: lambda self, e: self.naked_property(e),
exp.Stddev: rename_func("stddevSamp"),
}
@ -833,7 +887,7 @@ class ClickHouse(Dialect):
exp.OnCluster: exp.Properties.Location.POST_NAME,
}
# there's no list in docs, but it can be found in Clickhouse code
# There's no list in docs, but it can be found in Clickhouse code
# see `ClickHouse/src/Parsers/ParserCreate*.cpp`
ON_CLUSTER_TARGETS = {
"DATABASE",
@ -845,6 +899,14 @@ class ClickHouse(Dialect):
"NAMED COLLECTION",
}
# https://clickhouse.com/docs/en/sql-reference/data-types/nullable
NON_NULLABLE_TYPES = {
exp.DataType.Type.ARRAY,
exp.DataType.Type.MAP,
exp.DataType.Type.NULLABLE,
exp.DataType.Type.STRUCT,
}
def strtodate_sql(self, expression: exp.StrToDate) -> str:
strtodate_sql = self.function_fallback_sql(expression)
@ -863,6 +925,14 @@ class ClickHouse(Dialect):
return super().cast_sql(expression, safe_prefix=safe_prefix)
def trycast_sql(self, expression: exp.TryCast) -> str:
dtype = expression.to
if not dtype.is_type(*self.NON_NULLABLE_TYPES, check_nullable=True):
# Casting x into Nullable(T) appears to behave similarly to TRY_CAST(x AS T)
dtype.set("nullable", True)
return super().cast_sql(expression)
def _jsonpathsubscript_sql(self, expression: exp.JSONPathSubscript) -> str:
this = self.json_path_part(expression.this)
return str(int(this) + 1) if is_int(this) else this
@ -904,9 +974,30 @@ class ClickHouse(Dialect):
#
# https://clickhouse.com/docs/en/sql-reference/data-types/string
if expression.this in self.STRING_TYPE_MAPPING:
return "String"
dtype = "String"
else:
dtype = super().datatype_sql(expression)
return super().datatype_sql(expression)
# This section changes the type to `Nullable(...)` if the following conditions hold:
# - It's marked as nullable - this ensures we won't wrap ClickHouse types with `Nullable`
# and change their semantics
# - It's not the key type of a `Map`. This is because ClickHouse enforces the following
# constraint: "Type of Map key must be a type, that can be represented by integer or
# String or FixedString (possibly LowCardinality) or UUID or IPv6"
# - It's not a composite type, e.g. `Nullable(Array(...))` is not a valid type
parent = expression.parent
if (
expression.args.get("nullable") is not False
and not (
isinstance(parent, exp.DataType)
and parent.is_type(exp.DataType.Type.MAP, check_nullable=True)
and expression.index in (None, 0)
)
and not expression.is_type(*self.NON_NULLABLE_TYPES, check_nullable=True)
):
dtype = f"Nullable({dtype})"
return dtype
def cte_sql(self, expression: exp.CTE) -> str:
if expression.args.get("scalar"):
@ -953,7 +1044,10 @@ class ClickHouse(Dialect):
if expression.kind in self.ON_CLUSTER_TARGETS and locations.get(
exp.Properties.Location.POST_NAME
):
this_name = self.sql(expression.this, "this")
this_name = self.sql(
expression.this if isinstance(expression.this, exp.Schema) else expression,
"this",
)
this_properties = " ".join(
[self.sql(prop) for prop in locations[exp.Properties.Location.POST_NAME]]
)
@ -962,6 +1056,24 @@ class ClickHouse(Dialect):
return super().createable_sql(expression, locations)
def create_sql(self, expression: exp.Create) -> str:
# The comment property comes last in CTAS statements, i.e. after the query
query = expression.expression
if isinstance(query, exp.Query):
comment_prop = expression.find(exp.SchemaCommentProperty)
if comment_prop:
comment_prop.pop()
query.replace(exp.paren(query))
else:
comment_prop = None
create_sql = super().create_sql(expression)
comment_sql = self.sql(comment_prop)
comment_sql = f" {comment_sql}" if comment_sql else ""
return f"{create_sql}{comment_sql}"
def prewhere_sql(self, expression: exp.PreWhere) -> str:
this = self.indent(self.sql(expression, "this"))
return f"{self.seg('PREWHERE')}{self.sep()}{this}"

View file

@ -133,6 +133,10 @@ class _Dialect(type):
klass.INVERSE_FORMAT_MAPPING = {v: k for k, v in klass.FORMAT_MAPPING.items()}
klass.INVERSE_FORMAT_TRIE = new_trie(klass.INVERSE_FORMAT_MAPPING)
klass.INVERSE_CREATABLE_KIND_MAPPING = {
v: k for k, v in klass.CREATABLE_KIND_MAPPING.items()
}
base = seq_get(bases, 0)
base_tokenizer = (getattr(base, "tokenizer_class", Tokenizer),)
base_jsonpath_tokenizer = (getattr(base, "jsonpath_tokenizer_class", JSONPathTokenizer),)
@ -183,6 +187,9 @@ class _Dialect(type):
if enum not in ("", "bigquery"):
klass.generator_class.SELECT_KINDS = ()
if enum not in ("", "clickhouse"):
klass.generator_class.SUPPORTS_NULLABLE_TYPES = False
if enum not in ("", "athena", "presto", "trino"):
klass.generator_class.TRY_SUPPORTED = False
klass.generator_class.SUPPORTS_UESCAPE = False
@ -369,6 +376,24 @@ class Dialect(metaclass=_Dialect):
Whether ORDER BY ALL is supported (expands to all the selected columns) as in DuckDB, Spark3/Databricks
"""
HAS_DISTINCT_ARRAY_CONSTRUCTORS = False
"""
Whether the ARRAY constructor is context-sensitive, i.e in Redshift ARRAY[1, 2, 3] != ARRAY(1, 2, 3)
as the former is of type INT[] vs the latter which is SUPER
"""
SUPPORTS_FIXED_SIZE_ARRAYS = False
"""
Whether expressions such as x::INT[5] should be parsed as fixed-size array defs/casts e.g. in DuckDB. In
dialects which don't support fixed size arrays such as Snowflake, this should be interpreted as a subscript/index operator
"""
CREATABLE_KIND_MAPPING: dict[str, str] = {}
"""
Helper for dialects that use a different name for the same creatable kind. For example, the Clickhouse
equivalent of CREATE SCHEMA is CREATE DATABASE.
"""
# --- Autofilled ---
tokenizer_class = Tokenizer
@ -385,6 +410,8 @@ class Dialect(metaclass=_Dialect):
INVERSE_FORMAT_MAPPING: t.Dict[str, str] = {}
INVERSE_FORMAT_TRIE: t.Dict = {}
INVERSE_CREATABLE_KIND_MAPPING: dict[str, str] = {}
ESCAPED_SEQUENCES: t.Dict[str, str] = {}
# Delimiters for string literals and identifiers
@ -635,6 +662,9 @@ class Dialect(metaclass=_Dialect):
exp.GenerateDateArray: lambda self, e: self._annotate_with_type(
e, exp.DataType.build("ARRAY<DATE>")
),
exp.GenerateTimestampArray: lambda self, e: self._annotate_with_type(
e, exp.DataType.build("ARRAY<TIMESTAMP>")
),
exp.If: lambda self, e: self._annotate_by_args(e, "true", "false"),
exp.Interval: lambda self, e: self._annotate_with_type(e, exp.DataType.Type.INTERVAL),
exp.Least: lambda self, e: self._annotate_by_args(e, "expressions"),
@ -1214,7 +1244,13 @@ def right_to_substring_sql(self: Generator, expression: exp.Left) -> str:
def timestrtotime_sql(self: Generator, expression: exp.TimeStrToTime) -> str:
return self.sql(exp.cast(expression.this, exp.DataType.Type.TIMESTAMP))
datatype = (
exp.DataType.Type.TIMESTAMPTZ
if expression.args.get("zone")
else exp.DataType.Type.TIMESTAMP
)
return self.sql(exp.cast(expression.this, datatype, dialect=self.dialect))
def datestrtodate_sql(self: Generator, expression: exp.DateStrToDate) -> str:
@ -1464,7 +1500,12 @@ def merge_without_target_sql(self: Generator, expression: exp.Merge) -> str:
targets.add(normalize(alias.this))
for when in expression.expressions:
when.transform(
# only remove the target names from the THEN clause
# theyre still valid in the <condition> part of WHEN MATCHED / WHEN NOT MATCHED
# ref: https://github.com/TobikoData/sqlmesh/issues/2934
then = when.args.get("then")
if then:
then.transform(
lambda node: (
exp.column(node.this)
if isinstance(node, exp.Column) and normalize(node.args.get("table")) in targets
@ -1590,9 +1631,9 @@ def sha256_sql(self: Generator, expression: exp.SHA2) -> str:
return self.func(f"SHA{expression.text('length') or '256'}", expression.this)
def sequence_sql(self: Generator, expression: exp.GenerateSeries):
start = expression.args["start"]
end = expression.args["end"]
def sequence_sql(self: Generator, expression: exp.GenerateSeries | exp.GenerateDateArray) -> str:
start = expression.args.get("start")
end = expression.args.get("end")
step = expression.args.get("step")
if isinstance(start, exp.Cast):
@ -1602,8 +1643,8 @@ def sequence_sql(self: Generator, expression: exp.GenerateSeries):
else:
target_type = None
if target_type and target_type.is_type("timestamp"):
if target_type is start.to:
if start and end and target_type and target_type.is_type("date", "timestamp"):
if isinstance(start, exp.Cast) and target_type is start.to:
end = exp.cast(end, target_type)
else:
start = exp.cast(start, target_type)

View file

@ -32,6 +32,7 @@ from sqlglot.dialects.dialect import (
timestamptrunc_sql,
timestrtotime_sql,
unit_to_var,
unit_to_str,
)
from sqlglot.helper import seq_get
from sqlglot.tokens import TokenType
@ -81,6 +82,16 @@ def _date_sql(self: DuckDB.Generator, expression: exp.Date) -> str:
return result
# BigQuery -> DuckDB conversion for the TIME_DIFF function
def _timediff_sql(self: DuckDB.Generator, expression: exp.TimeDiff) -> str:
this = exp.cast(expression.this, exp.DataType.Type.TIME)
expr = exp.cast(expression.expression, exp.DataType.Type.TIME)
# Although the 2 dialects share similar signatures, BQ seems to inverse
# the sign of the result so the start/end time operands are flipped
return self.func("DATE_DIFF", unit_to_str(expression), expr, this)
def _array_sort_sql(self: DuckDB.Generator, expression: exp.ArraySort) -> str:
if expression.expression:
self.unsupported("DuckDB ARRAY_SORT does not support a comparator")
@ -160,8 +171,10 @@ def _datatype_sql(self: DuckDB.Generator, expression: exp.DataType) -> str:
if expression.is_type("array"):
return f"{self.expressions(expression, flat=True)}[{self.expressions(expression, key='values', flat=True)}]"
# Type TIMESTAMP / TIME WITH TIME ZONE does not support any modifiers
if expression.is_type("timestamptz", "timetz"):
# Modifiers are not supported for TIME, [TIME | TIMESTAMP] WITH TIME ZONE
if expression.is_type(
exp.DataType.Type.TIME, exp.DataType.Type.TIMETZ, exp.DataType.Type.TIMESTAMPTZ
):
return expression.this.value
return self.datatype_sql(expression)
@ -198,6 +211,41 @@ def _arrow_json_extract_sql(self: DuckDB.Generator, expression: JSON_EXTRACT_TYP
return arrow_sql
def _implicit_datetime_cast(
arg: t.Optional[exp.Expression], type: exp.DataType.Type = exp.DataType.Type.DATE
) -> t.Optional[exp.Expression]:
return exp.cast(arg, type) if isinstance(arg, exp.Literal) else arg
def _date_diff_sql(self: DuckDB.Generator, expression: exp.DateDiff) -> str:
this = _implicit_datetime_cast(expression.this)
expr = _implicit_datetime_cast(expression.expression)
return self.func("DATE_DIFF", unit_to_str(expression), expr, this)
def _generate_datetime_array_sql(
self: DuckDB.Generator, expression: t.Union[exp.GenerateDateArray, exp.GenerateTimestampArray]
) -> str:
is_generate_date_array = isinstance(expression, exp.GenerateDateArray)
type = exp.DataType.Type.DATE if is_generate_date_array else exp.DataType.Type.TIMESTAMP
start = _implicit_datetime_cast(expression.args.get("start"), type=type)
end = _implicit_datetime_cast(expression.args.get("end"), type=type)
# BQ's GENERATE_DATE_ARRAY & GENERATE_TIMESTAMP_ARRAY are transformed to DuckDB'S GENERATE_SERIES
gen_series: t.Union[exp.GenerateSeries, exp.Cast] = exp.GenerateSeries(
start=start, end=end, step=expression.args.get("step")
)
if is_generate_date_array:
# The GENERATE_SERIES result type is TIMESTAMP array, so to match BQ's semantics for
# GENERATE_DATE_ARRAY we must cast it back to DATE array
gen_series = exp.cast(gen_series, exp.DataType.build("ARRAY<DATE>"))
return self.sql(gen_series)
class DuckDB(Dialect):
NULL_ORDERING = "nulls_are_last"
SUPPORTS_USER_DEFINED_TYPES = False
@ -205,6 +253,7 @@ class DuckDB(Dialect):
INDEX_OFFSET = 1
CONCAT_COALESCE = True
SUPPORTS_ORDER_BY_ALL = True
SUPPORTS_FIXED_SIZE_ARRAYS = True
# https://duckdb.org/docs/sql/introduction.html#creating-a-new-table
NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
@ -293,7 +342,7 @@ class DuckDB(Dialect):
"LIST_HAS": exp.ArrayContains.from_arg_list,
"LIST_REVERSE_SORT": _build_sort_array_desc,
"LIST_SORT": exp.SortArray.from_arg_list,
"LIST_VALUE": exp.Array.from_arg_list,
"LIST_VALUE": lambda args: exp.Array(expressions=args),
"MAKE_TIME": exp.TimeFromParts.from_arg_list,
"MAKE_TIMESTAMP": _build_make_timestamp,
"MEDIAN": lambda args: exp.PercentileCont(
@ -416,6 +465,7 @@ class DuckDB(Dialect):
COPY_HAS_INTO_KEYWORD = False
STAR_EXCEPT = "EXCLUDE"
PAD_FILL_PATTERN_IS_REQUIRED = True
ARRAY_CONCAT_IS_VAR_LEN = False
TRANSFORMS = {
**generator.Generator.TRANSFORMS,
@ -441,9 +491,7 @@ class DuckDB(Dialect):
exp.DateAdd: _date_delta_sql,
exp.DateFromParts: rename_func("MAKE_DATE"),
exp.DateSub: _date_delta_sql,
exp.DateDiff: lambda self, e: self.func(
"DATE_DIFF", f"'{e.args.get('unit') or 'DAY'}'", e.expression, e.this
),
exp.DateDiff: _date_diff_sql,
exp.DateStrToDate: datestrtodate_sql,
exp.Datetime: no_datetime_sql,
exp.DatetimeSub: _date_delta_sql,
@ -454,6 +502,8 @@ class DuckDB(Dialect):
exp.DiToDate: lambda self,
e: f"CAST(STRPTIME(CAST({self.sql(e, 'this')} AS TEXT), {DuckDB.DATEINT_FORMAT}) AS DATE)",
exp.Encode: lambda self, e: encode_decode_sql(self, e, "ENCODE", replace=False),
exp.GenerateDateArray: _generate_datetime_array_sql,
exp.GenerateTimestampArray: _generate_datetime_array_sql,
exp.Explode: rename_func("UNNEST"),
exp.IntDiv: lambda self, e: self.binary(e, "//"),
exp.IsInf: rename_func("ISINF"),
@ -498,6 +548,7 @@ class DuckDB(Dialect):
exp.Struct: _struct_sql,
exp.TimeAdd: _date_delta_sql,
exp.Time: no_time_sql,
exp.TimeDiff: _timediff_sql,
exp.Timestamp: no_timestamp_sql,
exp.TimestampDiff: lambda self, e: self.func(
"DATE_DIFF", exp.Literal.string(e.unit), e.expression, e.this
@ -522,6 +573,9 @@ class DuckDB(Dialect):
exp.UnixToStr: lambda self, e: self.func(
"STRFTIME", self.func("TO_TIMESTAMP", e.this), self.format_time(e)
),
exp.DatetimeTrunc: lambda self, e: self.func(
"DATE_TRUNC", unit_to_str(e), exp.cast(e.this, exp.DataType.Type.DATETIME)
),
exp.UnixToTime: _unix_to_time_sql,
exp.UnixToTimeStr: lambda self, e: f"CAST(TO_TIMESTAMP({self.sql(e, 'this')}) AS TEXT)",
exp.VariancePop: rename_func("VAR_POP"),
@ -650,6 +704,9 @@ class DuckDB(Dialect):
PROPERTIES_LOCATION[exp.TemporaryProperty] = exp.Properties.Location.POST_CREATE
PROPERTIES_LOCATION[exp.ReturnsProperty] = exp.Properties.Location.POST_ALIAS
def fromiso8601timestamp_sql(self, expression: exp.FromISO8601Timestamp) -> str:
return self.sql(exp.cast(expression.this, exp.DataType.Type.TIMESTAMPTZ))
def strtotime_sql(self, expression: exp.StrToTime) -> str:
if expression.args.get("safe"):
formatted_time = self.format_time(expression)
@ -832,3 +889,24 @@ class DuckDB(Dialect):
return self.func("STRUCT_PACK", kv_sql)
return self.func("STRUCT_INSERT", this, kv_sql)
def unnest_sql(self, expression: exp.Unnest) -> str:
explode_array = expression.args.get("explode_array")
if explode_array:
# In BigQuery, UNNESTing a nested array leads to explosion of the top-level array & struct
# This is transpiled to DDB by transforming "FROM UNNEST(...)" to "FROM (SELECT UNNEST(..., max_depth => 2))"
expression.expressions.append(
exp.Kwarg(this=exp.var("max_depth"), expression=exp.Literal.number(2))
)
# If BQ's UNNEST is aliased, we transform it from a column alias to a table alias in DDB
alias = expression.args.get("alias")
if alias:
expression.set("alias", None)
alias = exp.TableAlias(this=seq_get(alias.args.get("columns"), 0))
unnest_sql = super().unnest_sql(expression)
select = exp.Select(expressions=[unnest_sql]).subquery(alias)
return self.sql(select)
return super().unnest_sql(expression)

View file

@ -252,6 +252,7 @@ class Hive(Dialect):
"ADD FILES": TokenType.COMMAND,
"ADD JAR": TokenType.COMMAND,
"ADD JARS": TokenType.COMMAND,
"MINUS": TokenType.EXCEPT,
"MSCK REPAIR": TokenType.COMMAND,
"REFRESH": TokenType.REFRESH,
"TIMESTAMP AS OF": TokenType.TIMESTAMP_SNAPSHOT,
@ -509,6 +510,7 @@ class Hive(Dialect):
e: f"STORED AS {self.sql(e, 'this') if isinstance(e.this, exp.InputOutputFormat) else e.name.upper()}",
exp.FromBase64: rename_func("UNBASE64"),
exp.GenerateSeries: sequence_sql,
exp.GenerateDateArray: sequence_sql,
exp.If: if_sql(),
exp.ILike: no_ilike_sql,
exp.IsNan: rename_func("ISNAN"),
@ -552,6 +554,7 @@ class Hive(Dialect):
exp.StrToTime: _str_to_time_sql,
exp.StrToUnix: _str_to_unix_sql,
exp.StructExtract: struct_extract_sql,
exp.Table: transforms.preprocess([transforms.unnest_generate_series]),
exp.TimeStrToDate: rename_func("TO_DATE"),
exp.TimeStrToTime: timestrtotime_sql,
exp.TimeStrToUnix: rename_func("UNIX_TIMESTAMP"),
@ -570,6 +573,7 @@ class Hive(Dialect):
),
exp.UnixToTime: _unix_to_time_sql,
exp.UnixToTimeStr: rename_func("FROM_UNIXTIME"),
exp.Unnest: rename_func("EXPLODE"),
exp.PartitionedByProperty: lambda self, e: f"PARTITIONED BY {self.sql(e, 'this')}",
exp.NumberToStr: rename_func("FORMAT_NUMBER"),
exp.National: lambda self, e: self.national_sql(e, prefix=""),
@ -593,6 +597,9 @@ class Hive(Dialect):
exp.WithDataProperty: exp.Properties.Location.UNSUPPORTED,
}
def unnest_sql(self, expression: exp.Unnest) -> str:
return rename_func("EXPLODE")(self, expression)
def _jsonpathkey_sql(self, expression: exp.JSONPathKey) -> str:
if isinstance(expression.this, exp.JSONPathWildcard):
self.unsupported("Unsupported wildcard in JSONPathKey expression")

View file

@ -302,6 +302,9 @@ class MySQL(Dialect):
FUNCTIONS = {
**parser.Parser.FUNCTIONS,
"CONVERT_TZ": lambda args: exp.ConvertTimezone(
source_tz=seq_get(args, 1), target_tz=seq_get(args, 2), timestamp=seq_get(args, 0)
),
"DATE": lambda args: exp.TsOrDsToDate(this=seq_get(args, 0)),
"DATE_ADD": build_date_delta_with_interval(exp.DateAdd),
"DATE_FORMAT": build_formatted_time(exp.TimeToStr, "mysql"),
@ -724,6 +727,7 @@ class MySQL(Dialect):
transforms.eliminate_semi_and_anti_joins,
transforms.eliminate_qualify,
transforms.eliminate_full_outer_join,
transforms.unnest_generate_date_array_using_recursive_cte,
]
),
exp.StrPosition: strposition_to_locate_sql,
@ -1213,3 +1217,10 @@ class MySQL(Dialect):
dateadd = build_date_delta_with_interval(exp.DateAdd)([start_ts, interval])
return self.sql(dateadd)
def converttimezone_sql(self, expression: exp.ConvertTimezone) -> str:
from_tz = expression.args.get("source_tz")
to_tz = expression.args.get("target_tz")
dt = expression.args.get("timestamp")
return self.func("CONVERT_TZ", dt, from_tz, to_tz)

View file

@ -78,6 +78,8 @@ class Oracle(Dialect):
for prefix in ("U", "u")
]
NESTED_COMMENTS = False
KEYWORDS = {
**tokens.Tokenizer.KEYWORDS,
"(+)": TokenType.JOIN_MARKER,
@ -90,7 +92,6 @@ class Oracle(Dialect):
"ORDER SIBLINGS BY": TokenType.ORDER_SIBLINGS_BY,
"SAMPLE": TokenType.TABLE_SAMPLE,
"START": TokenType.BEGIN,
"SYSDATE": TokenType.CURRENT_TIMESTAMP,
"TOP": TokenType.TOP,
"VARCHAR2": TokenType.VARCHAR,
}
@ -106,8 +107,15 @@ class Oracle(Dialect):
"TO_CHAR": _build_timetostr_or_tochar,
"TO_TIMESTAMP": build_formatted_time(exp.StrToTime, "oracle"),
"TO_DATE": build_formatted_time(exp.StrToDate, "oracle"),
"NVL": lambda args: exp.Coalesce(
this=seq_get(args, 0), expressions=args[1:], is_nvl=True
),
}
NO_PAREN_FUNCTION_PARSERS = {
**parser.Parser.NO_PAREN_FUNCTION_PARSERS,
"SYSDATE": lambda self: self.expression(exp.CurrentTimestamp, sysdate=True),
}
FUNCTIONS.pop("NVL")
FUNCTION_PARSERS: t.Dict[str, t.Callable] = {
**parser.Parser.FUNCTION_PARSERS,
@ -247,6 +255,7 @@ class Oracle(Dialect):
),
exp.Group: transforms.preprocess([transforms.unalias_group]),
exp.ILike: no_ilike_sql,
exp.Mod: rename_func("MOD"),
exp.Select: transforms.preprocess(
[
transforms.eliminate_distinct_on,
@ -274,6 +283,9 @@ class Oracle(Dialect):
}
def currenttimestamp_sql(self, expression: exp.CurrentTimestamp) -> str:
if expression.args.get("sysdate"):
return "SYSDATE"
this = expression.this
return self.func("CURRENT_TIMESTAMP", this) if this else "CURRENT_TIMESTAMP"
@ -291,7 +303,7 @@ class Oracle(Dialect):
)
return f"XMLTABLE({self.sep('')}{self.indent(this + passing + by_ref + columns)}{self.seg(')', sep='')}"
def add_column_sql(self, expression: exp.AlterTable) -> str:
def add_column_sql(self, expression: exp.Alter) -> str:
actions = self.expressions(expression, key="actions", flat=True)
if len(expression.args.get("actions", [])) > 1:
return f"ADD ({actions})"
@ -303,3 +315,7 @@ class Oracle(Dialect):
value = f" CONSTRAINT {value}" if value else ""
return f"{option}{value}"
def coalesce_sql(self, expression: exp.Coalesce) -> str:
func_name = "NVL" if expression.args.get("is_nvl") else "COALESCE"
return rename_func(func_name)(self, expression)

View file

@ -166,7 +166,7 @@ def _serial_to_generated(expression: exp.Expression) -> exp.Expression:
return expression
def _build_generate_series(args: t.List) -> exp.GenerateSeries:
def _build_generate_series(args: t.List) -> exp.ExplodingGenerateSeries:
# The goal is to convert step values like '1 day' or INTERVAL '1 day' into INTERVAL '1' day
# Note: postgres allows calls with just two arguments -- the "step" argument defaults to 1
step = seq_get(args, 2)
@ -176,7 +176,7 @@ def _build_generate_series(args: t.List) -> exp.GenerateSeries:
elif isinstance(step, exp.Interval) and not step.args.get("unit"):
args[2] = exp.to_interval(step.this.this)
return exp.GenerateSeries.from_arg_list(args)
return exp.ExplodingGenerateSeries.from_arg_list(args)
def _build_to_timestamp(args: t.List) -> exp.UnixToTime | exp.StrToTime:
@ -440,7 +440,7 @@ class Postgres(Dialect):
self._match(TokenType.COMMA)
value = self._parse_bitwise()
if part and part.is_string:
if part and isinstance(part, (exp.Column, exp.Literal)):
part = exp.var(part.name)
return self.expression(exp.Extract, this=part, expression=value)
@ -466,6 +466,7 @@ class Postgres(Dialect):
MULTI_ARG_DISTINCT = False
CAN_IMPLEMENT_ARRAY_ANY = True
COPY_HAS_INTO_KEYWORD = False
ARRAY_CONCAT_IS_VAR_LEN = False
SUPPORTED_JSON_PATH_PARTS = {
exp.JSONPathKey,
@ -487,12 +488,7 @@ class Postgres(Dialect):
TRANSFORMS = {
**generator.Generator.TRANSFORMS,
exp.AnyValue: any_value_to_max_sql,
exp.Array: lambda self, e: (
f"{self.normalize_func('ARRAY')}({self.sql(e.expressions[0])})"
if isinstance(seq_get(e.expressions, 0), exp.Select)
else f"{self.normalize_func('ARRAY')}[{self.expressions(e, flat=True)}]"
),
exp.ArrayConcat: rename_func("ARRAY_CAT"),
exp.ArrayConcat: lambda self, e: self.arrayconcat_sql(e, name="ARRAY_CAT"),
exp.ArrayContainsAll: lambda self, e: self.binary(e, "@>"),
exp.ArrayOverlaps: lambda self, e: self.binary(e, "&&"),
exp.ArrayFilter: filter_array_using_unnest,
@ -507,6 +503,7 @@ class Postgres(Dialect):
exp.DateStrToDate: datestrtodate_sql,
exp.DateSub: _date_add_sql("-"),
exp.Explode: rename_func("UNNEST"),
exp.ExplodingGenerateSeries: rename_func("GENERATE_SERIES"),
exp.GroupConcat: _string_agg_sql,
exp.IntDiv: rename_func("DIV"),
exp.JSONExtract: _json_extract_sql("JSON_EXTRACT_PATH", "->"),
@ -587,21 +584,32 @@ class Postgres(Dialect):
def unnest_sql(self, expression: exp.Unnest) -> str:
if len(expression.expressions) == 1:
arg = expression.expressions[0]
if isinstance(arg, exp.GenerateDateArray):
generate_series: exp.Expression = exp.GenerateSeries(**arg.args)
if isinstance(expression.parent, (exp.From, exp.Join)):
generate_series = (
exp.select("value::date")
.from_(generate_series.as_("value"))
.subquery(expression.args.get("alias") or "_unnested_generate_series")
)
return self.sql(generate_series)
from sqlglot.optimizer.annotate_types import annotate_types
this = annotate_types(expression.expressions[0])
this = annotate_types(arg)
if this.is_type("array<json>"):
while isinstance(this, exp.Cast):
this = this.this
arg = self.sql(exp.cast(this, exp.DataType.Type.JSON))
arg_as_json = self.sql(exp.cast(this, exp.DataType.Type.JSON))
alias = self.sql(expression, "alias")
alias = f" AS {alias}" if alias else ""
if expression.args.get("offset"):
self.unsupported("Unsupported JSON_ARRAY_ELEMENTS with offset")
return f"JSON_ARRAY_ELEMENTS({arg}){alias}"
return f"JSON_ARRAY_ELEMENTS({arg_as_json}){alias}"
return super().unnest_sql(expression)
@ -646,3 +654,11 @@ class Postgres(Dialect):
return self.sql(this)
return super().cast_sql(expression, safe_prefix=safe_prefix)
def array_sql(self, expression: exp.Array) -> str:
exprs = expression.expressions
return (
f"{self.normalize_func('ARRAY')}({self.sql(exprs[0])})"
if isinstance(seq_get(exprs, 0), exp.Select)
else f"{self.normalize_func('ARRAY')}[{self.expressions(expression, flat=True)}]"
)

View file

@ -142,17 +142,6 @@ def _build_from_unixtime(args: t.List) -> exp.Expression:
return exp.UnixToTime.from_arg_list(args)
def _unnest_sequence(expression: exp.Expression) -> exp.Expression:
if isinstance(expression, exp.Table):
if isinstance(expression.this, exp.GenerateSeries):
unnest = exp.Unnest(expressions=[expression.this])
if expression.alias:
return exp.alias_(unnest, alias="_u", table=[expression.alias], copy=False)
return unnest
return expression
def _first_last_sql(self: Presto.Generator, expression: exp.Func) -> str:
"""
Trino doesn't support FIRST / LAST as functions, but they're valid in the context
@ -245,13 +234,17 @@ class Presto(Dialect):
INDEX_OFFSET = 1
NULL_ORDERING = "nulls_are_last"
TIME_FORMAT = MySQL.TIME_FORMAT
TIME_MAPPING = MySQL.TIME_MAPPING
STRICT_STRING_CONCAT = True
SUPPORTS_SEMI_ANTI_JOIN = False
TYPED_DIVISION = True
TABLESAMPLE_SIZE_IS_PERCENT = True
LOG_BASE_FIRST: t.Optional[bool] = None
TIME_MAPPING = {
**MySQL.TIME_MAPPING,
"%W": "%A",
}
# https://github.com/trinodb/trino/issues/17
# https://github.com/trinodb/trino/issues/12289
# https://github.com/prestodb/presto/issues/2863
@ -434,6 +427,7 @@ class Presto(Dialect):
exp.FromTimeZone: lambda self,
e: f"WITH_TIMEZONE({self.sql(e, 'this')}, {self.sql(e, 'zone')}) AT TIME ZONE 'UTC'",
exp.GenerateSeries: sequence_sql,
exp.GenerateDateArray: sequence_sql,
exp.Group: transforms.preprocess([transforms.unalias_group]),
exp.GroupConcat: lambda self, e: self.func(
"ARRAY_JOIN", self.func("ARRAY_AGG", e.this), e.args.get("separator")
@ -471,7 +465,7 @@ class Presto(Dialect):
exp.StrToMap: rename_func("SPLIT_TO_MAP"),
exp.StrToTime: _str_to_time_sql,
exp.StructExtract: struct_extract_sql,
exp.Table: transforms.preprocess([_unnest_sequence]),
exp.Table: transforms.preprocess([transforms.unnest_generate_series]),
exp.Timestamp: no_timestamp_sql,
exp.TimestampAdd: _date_delta_sql("DATE_ADD"),
exp.TimestampTrunc: timestamptrunc_sql(),

View file

@ -17,6 +17,7 @@ from sqlglot.dialects.dialect import (
from sqlglot.dialects.postgres import Postgres
from sqlglot.helper import seq_get
from sqlglot.tokens import TokenType
from sqlglot.parser import build_convert_timezone
if t.TYPE_CHECKING:
from sqlglot._typing import E
@ -45,13 +46,11 @@ class Redshift(Postgres):
INDEX_OFFSET = 0
COPY_PARAMS_ARE_CSV = False
HEX_LOWERCASE = True
HAS_DISTINCT_ARRAY_CONSTRUCTORS = True
TIME_FORMAT = "'YYYY-MM-DD HH:MI:SS'"
TIME_MAPPING = {
**Postgres.TIME_MAPPING,
"MON": "%b",
"HH": "%H",
}
# ref: https://docs.aws.amazon.com/redshift/latest/dg/r_FORMAT_strings.html
TIME_FORMAT = "'YYYY-MM-DD HH24:MI:SS'"
TIME_MAPPING = {**Postgres.TIME_MAPPING, "MON": "%b", "HH24": "%H", "HH": "%I"}
class Parser(Postgres.Parser):
FUNCTIONS = {
@ -62,6 +61,7 @@ class Redshift(Postgres):
unit=exp.var("month"),
return_type=exp.DataType.build("TIMESTAMP"),
),
"CONVERT_TIMEZONE": lambda args: build_convert_timezone(args, "UTC"),
"DATEADD": _build_date_delta(exp.TsOrDsAdd),
"DATE_ADD": _build_date_delta(exp.TsOrDsAdd),
"DATEDIFF": _build_date_delta(exp.TsOrDsDiff),
@ -77,7 +77,7 @@ class Redshift(Postgres):
NO_PAREN_FUNCTION_PARSERS = {
**Postgres.Parser.NO_PAREN_FUNCTION_PARSERS,
"APPROXIMATE": lambda self: self._parse_approximate_count(),
"SYSDATE": lambda self: self.expression(exp.CurrentTimestamp, transaction=True),
"SYSDATE": lambda self: self.expression(exp.CurrentTimestamp, sysdate=True),
}
SUPPORTS_IMPLICIT_UNNEST = True
@ -153,6 +153,8 @@ class Redshift(Postgres):
COPY_PARAMS_ARE_WRAPPED = False
HEX_FUNC = "TO_HEX"
PARSE_JSON_NAME = "JSON_PARSE"
ARRAY_CONCAT_IS_VAR_LEN = False
SUPPORTS_CONVERT_TIMEZONE = True
# Redshift doesn't have `WITH` as part of their with_properties so we remove it
WITH_PROPERTIES_PREFIX = " "
@ -169,12 +171,13 @@ class Redshift(Postgres):
TRANSFORMS = {
**Postgres.Generator.TRANSFORMS,
exp.ArrayConcat: lambda self, e: self.arrayconcat_sql(e, name="ARRAY_CONCAT"),
exp.Concat: concat_to_dpipe_sql,
exp.ConcatWs: concat_ws_to_dpipe_sql,
exp.ApproxDistinct: lambda self,
e: f"APPROXIMATE COUNT(DISTINCT {self.sql(e, 'this')})",
exp.CurrentTimestamp: lambda self, e: (
"SYSDATE" if e.args.get("transaction") else "GETDATE()"
"SYSDATE" if e.args.get("sysdate") else "GETDATE()"
),
exp.DateAdd: date_delta_sql("DATEADD"),
exp.DateDiff: date_delta_sql("DATEDIFF"),
@ -191,6 +194,7 @@ class Redshift(Postgres):
transforms.eliminate_distinct_on,
transforms.eliminate_semi_and_anti_joins,
transforms.unqualify_unnest,
transforms.unnest_generate_date_array_using_recursive_cte,
]
),
exp.SortKeyProperty: lambda self,
@ -423,3 +427,9 @@ class Redshift(Postgres):
file_format = f" FILE FORMAT {file_format}" if file_format else ""
return f"SET{exprs}{location}{file_format}"
def array_sql(self, expression: exp.Array) -> str:
if expression.args.get("bracket_notation"):
return super().array_sql(expression)
return rename_func("ARRAY")(self, expression)

View file

@ -122,12 +122,6 @@ def _regexpilike_sql(self: Snowflake.Generator, expression: exp.RegexpILike) ->
)
def _build_convert_timezone(args: t.List) -> t.Union[exp.Anonymous, exp.AtTimeZone]:
if len(args) == 3:
return exp.Anonymous(this="CONVERT_TIMEZONE", expressions=args)
return exp.AtTimeZone(this=seq_get(args, 1), zone=seq_get(args, 0))
def _build_regexp_replace(args: t.List) -> exp.RegexpReplace:
regexp_replace = exp.RegexpReplace.from_arg_list(args)
@ -186,6 +180,47 @@ def _flatten_structured_types_unless_iceberg(expression: exp.Expression) -> exp.
return expression
def _unnest_generate_date_array(expression: exp.Expression) -> exp.Expression:
if isinstance(expression, exp.Select):
for unnest in expression.find_all(exp.Unnest):
if (
isinstance(unnest.parent, (exp.From, exp.Join))
and len(unnest.expressions) == 1
and isinstance(unnest.expressions[0], exp.GenerateDateArray)
):
generate_date_array = unnest.expressions[0]
start = generate_date_array.args.get("start")
end = generate_date_array.args.get("end")
step = generate_date_array.args.get("step")
if not start or not end or not isinstance(step, exp.Interval) or step.name != "1":
continue
unit = step.args.get("unit")
unnest_alias = unnest.args.get("alias")
if unnest_alias:
unnest_alias = unnest_alias.copy()
sequence_value_name = seq_get(unnest_alias.columns, 0) or "value"
else:
sequence_value_name = "value"
# We'll add the next sequence value to the starting date and project the result
date_add = _build_date_time_add(exp.DateAdd)(
[unit, exp.cast(sequence_value_name, "int"), exp.cast(start, "date")]
).as_(sequence_value_name)
# We use DATEDIFF to compute the number of sequence values needed
number_sequence = Snowflake.Parser.FUNCTIONS["ARRAY_GENERATE_RANGE"](
[exp.Literal.number(0), _build_datediff([unit, start, end]) + 1]
)
unnest.set("expressions", [number_sequence])
unnest.replace(exp.select(date_add).from_(unnest.copy()).subquery(unnest_alias))
return expression
class Snowflake(Dialect):
# https://docs.snowflake.com/en/sql-reference/identifiers-syntax
NORMALIZATION_STRATEGY = NormalizationStrategy.UPPERCASE
@ -255,7 +290,7 @@ class Snowflake(Dialect):
**parser.Parser.FUNCTIONS,
"APPROX_PERCENTILE": exp.ApproxQuantile.from_arg_list,
"ARRAYAGG": exp.ArrayAgg.from_arg_list,
"ARRAY_CONSTRUCT": exp.Array.from_arg_list,
"ARRAY_CONSTRUCT": lambda args: exp.Array(expressions=args),
"ARRAY_CONTAINS": lambda args: exp.ArrayContains(
this=seq_get(args, 1), expression=seq_get(args, 0)
),
@ -268,7 +303,6 @@ class Snowflake(Dialect):
"BITXOR": binary_from_function(exp.BitwiseXor),
"BIT_XOR": binary_from_function(exp.BitwiseXor),
"BOOLXOR": binary_from_function(exp.Xor),
"CONVERT_TIMEZONE": _build_convert_timezone,
"DATE": _build_datetime("DATE", exp.DataType.Type.DATE),
"DATE_TRUNC": _date_trunc_to_time,
"DATEADD": _build_date_time_add(exp.DateAdd),
@ -413,6 +447,26 @@ class Snowflake(Dialect):
),
}
def _negate_range(
self, this: t.Optional[exp.Expression] = None
) -> t.Optional[exp.Expression]:
if not this:
return this
query = this.args.get("query")
if isinstance(this, exp.In) and isinstance(query, exp.Query):
# Snowflake treats `value NOT IN (subquery)` as `VALUE <> ALL (subquery)`, so
# we do this conversion here to avoid parsing it into `NOT value IN (subquery)`
# which can produce different results (most likely a SnowFlake bug).
#
# https://docs.snowflake.com/en/sql-reference/functions/in
# Context: https://github.com/tobymao/sqlglot/issues/3890
return self.expression(
exp.NEQ, this=this.this, expression=exp.All(this=query.unnest())
)
return self.expression(exp.Not, this=this)
def _parse_with_constraint(self) -> t.Optional[exp.Expression]:
if self._prev.token_type != TokenType.WITH:
self._retreat(self._index - 1)
@ -638,6 +692,7 @@ class Snowflake(Dialect):
HEX_STRINGS = [("x'", "'"), ("X'", "'")]
RAW_STRINGS = ["$$"]
COMMENTS = ["--", "//", ("/*", "*/")]
NESTED_COMMENTS = False
KEYWORDS = {
**tokens.Tokenizer.KEYWORDS,
@ -692,6 +747,9 @@ class Snowflake(Dialect):
COPY_PARAMS_ARE_WRAPPED = False
COPY_PARAMS_EQ_REQUIRED = True
STAR_EXCEPT = "EXCLUDE"
SUPPORTS_EXPLODING_PROJECTIONS = False
ARRAY_CONCAT_IS_VAR_LEN = False
SUPPORTS_CONVERT_TIMEZONE = True
TRANSFORMS = {
**generator.Generator.TRANSFORMS,
@ -699,7 +757,7 @@ class Snowflake(Dialect):
exp.ArgMax: rename_func("MAX_BY"),
exp.ArgMin: rename_func("MIN_BY"),
exp.Array: inline_array_sql,
exp.ArrayConcat: rename_func("ARRAY_CAT"),
exp.ArrayConcat: lambda self, e: self.arrayconcat_sql(e, name="ARRAY_CAT"),
exp.ArrayContains: lambda self, e: self.func("ARRAY_CONTAINS", e.expression, e.this),
exp.AtTimeZone: lambda self, e: self.func(
"CONVERT_TIMEZONE", e.args.get("zone"), e.this
@ -751,6 +809,7 @@ class Snowflake(Dialect):
transforms.eliminate_distinct_on,
transforms.explode_to_unnest(),
transforms.eliminate_semi_and_anti_joins,
_unnest_generate_date_array,
]
),
exp.SHA: rename_func("SHA1"),

View file

@ -132,6 +132,7 @@ class Spark(Spark2):
class Generator(Spark2.Generator):
SUPPORTS_TO_NUMBER = True
PAD_FILL_PATTERN_IS_REQUIRED = False
SUPPORTS_CONVERT_TIMEZONE = True
TYPE_MAPPING = {
**Spark2.Generator.TYPE_MAPPING,

View file

@ -18,6 +18,7 @@ from sqlglot.dialects.dialect import (
build_date_delta,
rename_func,
trim_sql,
timestrtotime_sql,
)
from sqlglot.helper import seq_get
from sqlglot.time import format_time
@ -339,6 +340,16 @@ def _json_extract_sql(
return self.func("ISNULL", json_query, json_value)
def _timestrtotime_sql(self: TSQL.Generator, expression: exp.TimeStrToTime):
sql = timestrtotime_sql(self, expression)
if expression.args.get("zone"):
# If there is a timezone, produce an expression like:
# CAST('2020-01-01 12:13:14-08:00' AS DATETIMEOFFSET) AT TIME ZONE 'UTC'
# If you dont have AT TIME ZONE 'UTC', wrapping that expression in another cast back to DATETIME2 just drops the timezone information
return self.sql(exp.AtTimeZone(this=sql, zone=exp.Literal.string("UTC")))
return sql
class TSQL(Dialect):
NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
TIME_FORMAT = "'yyyy-mm-dd hh:mm:ss'"
@ -863,6 +874,7 @@ class TSQL(Dialect):
transforms.eliminate_distinct_on,
transforms.eliminate_semi_and_anti_joins,
transforms.eliminate_qualify,
transforms.unnest_generate_date_array_using_recursive_cte,
]
),
exp.Stddev: rename_func("STDEV"),
@ -875,9 +887,7 @@ class TSQL(Dialect):
"HASHBYTES", exp.Literal.string(f"SHA2_{e.args.get('length', 256)}"), e.this
),
exp.TemporaryProperty: lambda self, e: "",
exp.TimeStrToTime: lambda self, e: self.sql(
exp.cast(e.this, exp.DataType.Type.DATETIME)
),
exp.TimeStrToTime: _timestrtotime_sql,
exp.TimeToStr: _format_sql,
exp.Trim: trim_sql,
exp.TsOrDsAdd: date_delta_sql("DATEADD", cast=True),
@ -1139,11 +1149,11 @@ class TSQL(Dialect):
def partition_sql(self, expression: exp.Partition) -> str:
return f"WITH (PARTITIONS({self.expressions(expression, flat=True)}))"
def altertable_sql(self, expression: exp.AlterTable) -> str:
def alter_sql(self, expression: exp.Alter) -> str:
action = seq_get(expression.args.get("actions") or [], 0)
if isinstance(action, exp.RenameTable):
return f"EXEC sp_rename '{self.sql(expression.this)}', '{action.this.name}'"
return super().altertable_sql(expression)
return super().alter_sql(expression)
def drop_sql(self, expression: exp.Drop) -> str:
if expression.args["kind"] == "VIEW":

View file

@ -74,7 +74,9 @@ def execute(
raise ExecuteError("Tables must support the same table args as schema")
now = time.time()
expression = optimize(sql, schema, leave_tables_isolated=True, dialect=read)
expression = optimize(
sql, schema, leave_tables_isolated=True, infer_csv_schemas=True, dialect=read
)
logger.debug("Optimization finished: %f", time.time() - now)
logger.debug("Optimized SQL: %s", expression.sql(pretty=True))

View file

@ -1387,6 +1387,7 @@ class Create(DDL):
"exists": False,
"properties": False,
"replace": False,
"refresh": False,
"unique": False,
"indexes": False,
"no_schema_binding": False,
@ -1436,7 +1437,13 @@ class Clone(Expression):
class Describe(Expression):
arg_types = {"this": True, "style": False, "kind": False, "expressions": False}
arg_types = {
"this": True,
"style": False,
"kind": False,
"expressions": False,
"partition": False,
}
# https://duckdb.org/docs/guides/meta/summarize.html
@ -2000,6 +2007,11 @@ class Drop(Expression):
"cluster": False,
}
@property
def kind(self) -> t.Optional[str]:
kind = self.args.get("kind")
return kind and kind.upper()
class Filter(Expression):
arg_types = {"this": True, "expression": True}
@ -2158,6 +2170,8 @@ class Insert(DDL, DML):
"ignore": False,
"by_name": False,
"stored": False,
"partition": False,
"settings": False,
}
def with_(
@ -2464,17 +2478,17 @@ class Offset(Expression):
class Order(Expression):
arg_types = {
"this": False,
"expressions": True,
"interpolate": False,
"siblings": False,
}
arg_types = {"this": False, "expressions": True, "siblings": False}
# https://clickhouse.com/docs/en/sql-reference/statements/select/order-by#order-by-expr-with-fill-modifier
class WithFill(Expression):
arg_types = {"from": False, "to": False, "step": False}
arg_types = {
"from": False,
"to": False,
"step": False,
"interpolate": False,
}
# hive specific sorts
@ -2669,6 +2683,11 @@ class OnCluster(Property):
arg_types = {"this": True}
# Clickhouse EMPTY table "property"
class EmptyProperty(Property):
arg_types = {}
class LikeProperty(Property):
arg_types = {"this": True, "expressions": False}
@ -2735,6 +2754,10 @@ class PartitionedOfProperty(Property):
arg_types = {"this": True, "expression": True}
class StreamingTableProperty(Property):
arg_types = {}
class RemoteWithConnectionModelProperty(Property):
arg_types = {"this": True}
@ -3137,11 +3160,11 @@ class SetOperation(Query):
return self.this.unnest().selects
@property
def left(self) -> Expression:
def left(self) -> Query:
return self.this
@property
def right(self) -> Expression:
def right(self) -> Query:
return self.expression
@ -3859,6 +3882,7 @@ class Pivot(Expression):
"group": False,
"columns": False,
"include_nulls": False,
"default_on_null": False,
}
@property
@ -3948,6 +3972,8 @@ class DataTypeParam(Expression):
return self.this.name
# The `nullable` arg is helpful when transpiling types from other dialects to ClickHouse, which
# assumes non-nullable types by default. Values `None` and `True` mean the type is nullable.
class DataType(Expression):
arg_types = {
"this": True,
@ -3956,6 +3982,7 @@ class DataType(Expression):
"values": False,
"prefix": False,
"kind": False,
"nullable": False,
}
class Type(AutoName):
@ -4194,28 +4221,45 @@ class DataType(Expression):
return DataType(**{**data_type_exp.args, **kwargs})
def is_type(self, *dtypes: DATA_TYPE) -> bool:
def is_type(self, *dtypes: DATA_TYPE, check_nullable: bool = False) -> bool:
"""
Checks whether this DataType matches one of the provided data types. Nested types or precision
will be compared using "structural equivalence" semantics, so e.g. array<int> != array<float>.
Args:
dtypes: the data types to compare this DataType to.
check_nullable: whether to take the NULLABLE type constructor into account for the comparison.
If false, it means that NULLABLE<INT> is equivalent to INT.
Returns:
True, if and only if there is a type in `dtypes` which is equal to this DataType.
"""
if (
not check_nullable
and self.this == DataType.Type.NULLABLE
and len(self.expressions) == 1
):
this_type = self.expressions[0]
else:
this_type = self
for dtype in dtypes:
other = DataType.build(dtype, copy=False, udt=True)
other_type = DataType.build(dtype, copy=False, udt=True)
if (
not check_nullable
and other_type.this == DataType.Type.NULLABLE
and len(other_type.expressions) == 1
):
other_type = other_type.expressions[0]
if (
other.expressions
or self.this == DataType.Type.USERDEFINED
or other.this == DataType.Type.USERDEFINED
other_type.expressions
or this_type.this == DataType.Type.USERDEFINED
or other_type.this == DataType.Type.USERDEFINED
):
matches = self == other
matches = this_type == other_type
else:
matches = self.this == other.this
matches = this_type.this == other_type.this
if matches:
return True
@ -4270,9 +4314,10 @@ class Rollback(Expression):
arg_types = {"savepoint": False, "this": False}
class AlterTable(Expression):
class Alter(Expression):
arg_types = {
"this": True,
"kind": True,
"actions": True,
"exists": False,
"only": False,
@ -4536,6 +4581,12 @@ class PivotAlias(Alias):
pass
# Represents Snowflake's ANY [ ORDER BY ... ] syntax
# https://docs.snowflake.com/en/sql-reference/constructs/pivot
class PivotAny(Expression):
arg_types = {"this": False}
class Aliases(Expression):
arg_types = {"this": True, "expressions": True}
@ -4790,7 +4841,7 @@ class ApproxDistinct(AggFunc):
class Array(Func):
arg_types = {"expressions": False}
arg_types = {"expressions": False, "bracket_notation": False}
is_var_len_args = True
@ -4833,10 +4884,21 @@ class Convert(Func):
arg_types = {"this": True, "expression": True, "style": False}
class ConvertTimezone(Func):
arg_types = {"source_tz": False, "target_tz": True, "timestamp": True}
class GenerateSeries(Func):
arg_types = {"start": True, "end": True, "step": False, "is_end_exclusive": False}
# Postgres' GENERATE_SERIES function returns a row set, i.e. it implicitly explodes when it's
# used in a projection, so this expression is a helper that facilitates transpilation to other
# dialects. For example, we'd generate UNNEST(GENERATE_SERIES(...)) in DuckDB
class ExplodingGenerateSeries(GenerateSeries):
pass
class ArrayAgg(AggFunc):
pass
@ -5025,7 +5087,7 @@ class Ceil(Func):
class Coalesce(Func):
arg_types = {"this": True, "expressions": False}
arg_types = {"this": True, "expressions": False, "is_nvl": False}
is_var_len_args = True
_sql_names = ["COALESCE", "IFNULL", "NVL"]
@ -5077,7 +5139,7 @@ class CurrentTime(Func):
class CurrentTimestamp(Func):
arg_types = {"this": False, "transaction": False}
arg_types = {"this": False, "sysdate": False}
class CurrentUser(Func):
@ -5286,6 +5348,7 @@ class Unnest(Func, UDTF):
"expressions": True,
"alias": False,
"offset": False,
"explode_array": False,
}
@property
@ -5309,6 +5372,11 @@ class ToBase64(Func):
pass
# https://trino.io/docs/current/functions/datetime.html#from_iso8601_timestamp
class FromISO8601Timestamp(Func):
_sql_names = ["FROM_ISO8601_TIMESTAMP"]
class GapFill(Func):
arg_types = {
"this": True,
@ -5321,8 +5389,14 @@ class GapFill(Func):
}
# https://cloud.google.com/bigquery/docs/reference/standard-sql/array_functions#generate_date_array
class GenerateDateArray(Func):
arg_types = {"start": True, "end": True, "interval": False}
arg_types = {"start": True, "end": True, "step": False}
# https://cloud.google.com/bigquery/docs/reference/standard-sql/array_functions#generate_timestamp_array
class GenerateTimestampArray(Func):
arg_types = {"start": True, "end": True, "step": True}
class Greatest(Func):
@ -5639,6 +5713,10 @@ class ScopeResolution(Expression):
arg_types = {"this": False, "expression": True}
class Stream(Expression):
pass
class StarMap(Func):
pass
@ -5920,7 +5998,7 @@ class Time(Func):
class TimeToStr(Func):
arg_types = {"this": True, "format": True, "culture": False, "timezone": False}
arg_types = {"this": True, "format": True, "culture": False, "zone": False}
class TimeToTimeStr(Func):
@ -5936,7 +6014,7 @@ class TimeStrToDate(Func):
class TimeStrToTime(Func):
pass
arg_types = {"this": True, "zone": False}
class TimeStrToUnix(Func):
@ -7144,7 +7222,9 @@ def column(
return this
def cast(expression: ExpOrStr, to: DATA_TYPE, copy: bool = True, **opts) -> Cast:
def cast(
expression: ExpOrStr, to: DATA_TYPE, copy: bool = True, dialect: DialectType = None, **opts
) -> Cast:
"""Cast an expression to a data type.
Example:
@ -7155,14 +7235,36 @@ def cast(expression: ExpOrStr, to: DATA_TYPE, copy: bool = True, **opts) -> Cast
expression: The expression to cast.
to: The datatype to cast to.
copy: Whether to copy the supplied expressions.
dialect: The target dialect. This is used to prevent a re-cast in the following scenario:
- The expression to be cast is already a exp.Cast expression
- The existing cast is to a type that is logically equivalent to new type
For example, if :expression='CAST(x as DATETIME)' and :to=Type.TIMESTAMP,
but in the target dialect DATETIME is mapped to TIMESTAMP, then we will NOT return `CAST(x (as DATETIME) as TIMESTAMP)`
and instead just return the original expression `CAST(x as DATETIME)`.
This is to prevent it being output as a double cast `CAST(x (as TIMESTAMP) as TIMESTAMP)` once the DATETIME -> TIMESTAMP
mapping is applied in the target dialect generator.
Returns:
The new Cast instance.
"""
expr = maybe_parse(expression, copy=copy, **opts)
data_type = DataType.build(to, copy=copy, **opts)
expr = maybe_parse(expression, copy=copy, dialect=dialect, **opts)
data_type = DataType.build(to, copy=copy, dialect=dialect, **opts)
if expr.is_type(data_type):
# dont re-cast if the expression is already a cast to the correct type
if isinstance(expr, Cast):
from sqlglot.dialects.dialect import Dialect
target_dialect = Dialect.get_or_raise(dialect)
type_mapping = target_dialect.generator_class.TYPE_MAPPING
existing_cast_type: DataType.Type = expr.to.this
new_cast_type: DataType.Type = data_type.this
types_are_equivalent = type_mapping.get(
existing_cast_type, existing_cast_type
) == type_mapping.get(new_cast_type, new_cast_type)
if expr.is_type(data_type) or types_are_equivalent:
return expr
expr = Cast(this=expr, to=data_type)
@ -7259,7 +7361,7 @@ def rename_table(
old_name: str | Table,
new_name: str | Table,
dialect: DialectType = None,
) -> AlterTable:
) -> Alter:
"""Build ALTER TABLE... RENAME... expression
Args:
@ -7272,8 +7374,9 @@ def rename_table(
"""
old_table = to_table(old_name, dialect=dialect)
new_table = to_table(new_name, dialect=dialect)
return AlterTable(
return Alter(
this=old_table,
kind="TABLE",
actions=[
RenameTable(this=new_table),
],
@ -7286,7 +7389,7 @@ def rename_column(
new_column_name: str | Column,
exists: t.Optional[bool] = None,
dialect: DialectType = None,
) -> AlterTable:
) -> Alter:
"""Build ALTER TABLE... RENAME COLUMN... expression
Args:
@ -7302,8 +7405,9 @@ def rename_column(
table = to_table(table_name, dialect=dialect)
old_column = to_column(old_column_name, dialect=dialect)
new_column = to_column(new_column_name, dialect=dialect)
return AlterTable(
return Alter(
this=table,
kind="TABLE",
actions=[
RenameColumn(this=old_column, to=new_column, exists=exists),
],
@ -7335,12 +7439,15 @@ def convert(value: t.Any, copy: bool = False) -> Expression:
if isinstance(value, bytes):
return HexString(this=value.hex())
if isinstance(value, datetime.datetime):
datetime_literal = Literal.string(
(value if value.tzinfo else value.replace(tzinfo=datetime.timezone.utc)).isoformat(
sep=" "
)
)
return TimeStrToTime(this=datetime_literal)
datetime_literal = Literal.string(value.isoformat(sep=" "))
tz = None
if value.tzinfo:
# this works for zoneinfo.ZoneInfo, pytz.timezone and datetime.datetime.utc to return IANA timezone names like "America/Los_Angeles"
# instead of abbreviations like "PDT". This is for consistency with other timezone handling functions in SQLGlot
tz = Literal.string(str(value.tzinfo))
return TimeStrToTime(this=datetime_literal, zone=tz)
if isinstance(value, datetime.date):
date_literal = Literal.string(value.strftime("%Y-%m-%d"))
return DateStrToDate(this=date_literal)

View file

@ -92,6 +92,7 @@ class Generator(metaclass=_Generator):
exp.DateFormatColumnConstraint: lambda self, e: f"FORMAT {self.sql(e, 'this')}",
exp.DefaultColumnConstraint: lambda self, e: f"DEFAULT {self.sql(e, 'this')}",
exp.DynamicProperty: lambda *_: "DYNAMIC",
exp.EmptyProperty: lambda *_: "EMPTY",
exp.EncodeColumnConstraint: lambda self, e: f"ENCODE {self.sql(e, 'this')}",
exp.EphemeralColumnConstraint: lambda self,
e: f"EPHEMERAL{(' ' + self.sql(e, 'this')) if e.this else ''}",
@ -117,8 +118,10 @@ class Generator(metaclass=_Generator):
e: f"ON COMMIT {'DELETE' if e.args.get('delete') else 'PRESERVE'} ROWS",
exp.OnProperty: lambda self, e: f"ON {self.sql(e, 'this')}",
exp.OnUpdateColumnConstraint: lambda self, e: f"ON UPDATE {self.sql(e, 'this')}",
exp.Operator: lambda self, e: self.binary(e, ""), # The operator is produced in `binary`
exp.OutputModelProperty: lambda self, e: f"OUTPUT{self.sql(e, 'this')}",
exp.PathColumnConstraint: lambda self, e: f"PATH {self.sql(e, 'this')}",
exp.PivotAny: lambda self, e: f"ANY{self.sql(e, 'this')}",
exp.ProjectionPolicyColumnConstraint: lambda self,
e: f"PROJECTION POLICY {self.sql(e, 'this')}",
exp.RemoteWithConnectionModelProperty: lambda self,
@ -136,6 +139,8 @@ class Generator(metaclass=_Generator):
exp.SqlSecurityProperty: lambda _,
e: f"SQL SECURITY {'DEFINER' if e.args.get('definer') else 'INVOKER'}",
exp.StabilityProperty: lambda _, e: e.name,
exp.Stream: lambda self, e: f"STREAM {self.sql(e, 'this')}",
exp.StreamingTableProperty: lambda *_: "STREAMING",
exp.StrictProperty: lambda *_: "STRICT",
exp.TemporaryProperty: lambda *_: "TEMPORARY",
exp.TagColumnConstraint: lambda self, e: f"TAG ({self.expressions(e, flat=True)})",
@ -371,6 +376,18 @@ class Generator(metaclass=_Generator):
# Whether the text pattern/fill (3rd) parameter of RPAD()/LPAD() is optional (defaults to space)
PAD_FILL_PATTERN_IS_REQUIRED = False
# Whether a projection can explode into multiple rows, e.g. by unnesting an array.
SUPPORTS_EXPLODING_PROJECTIONS = True
# Whether ARRAY_CONCAT can be generated with varlen args or if it should be reduced to 2-arg version
ARRAY_CONCAT_IS_VAR_LEN = True
# Whether CONVERT_TIMEZONE() is supported; if not, it will be generated as exp.AtTimeZone
SUPPORTS_CONVERT_TIMEZONE = False
# Whether nullable types can be constructed, e.g. `Nullable(Int64)`
SUPPORTS_NULLABLE_TYPES = True
# The name to generate for the JSONPath expression. If `None`, only `this` will be generated
PARSE_JSON_NAME: t.Optional[str] = "PARSE_JSON"
@ -439,6 +456,7 @@ class Generator(metaclass=_Generator):
exp.DynamicProperty: exp.Properties.Location.POST_CREATE,
exp.DistKeyProperty: exp.Properties.Location.POST_SCHEMA,
exp.DistStyleProperty: exp.Properties.Location.POST_SCHEMA,
exp.EmptyProperty: exp.Properties.Location.POST_SCHEMA,
exp.EngineProperty: exp.Properties.Location.POST_SCHEMA,
exp.ExecuteAsProperty: exp.Properties.Location.POST_SCHEMA,
exp.ExternalProperty: exp.Properties.Location.POST_CREATE,
@ -488,6 +506,7 @@ class Generator(metaclass=_Generator):
exp.SqlReadWriteProperty: exp.Properties.Location.POST_SCHEMA,
exp.SqlSecurityProperty: exp.Properties.Location.POST_CREATE,
exp.StabilityProperty: exp.Properties.Location.POST_SCHEMA,
exp.StreamingTableProperty: exp.Properties.Location.POST_CREATE,
exp.StrictProperty: exp.Properties.Location.POST_SCHEMA,
exp.TemporaryProperty: exp.Properties.Location.POST_CREATE,
exp.ToTableProperty: exp.Properties.Location.POST_SCHEMA,
@ -962,6 +981,7 @@ class Generator(metaclass=_Generator):
def create_sql(self, expression: exp.Create) -> str:
kind = self.sql(expression, "kind")
kind = self.dialect.INVERSE_CREATABLE_KIND_MAPPING.get(kind) or kind
properties = expression.args.get("properties")
properties_locs = self.locate_properties(properties) if properties else defaultdict()
@ -1018,6 +1038,7 @@ class Generator(metaclass=_Generator):
index_sql = indexes + postindex_props_sql
replace = " OR REPLACE" if expression.args.get("replace") else ""
refresh = " OR REFRESH" if expression.args.get("refresh") else ""
unique = " UNIQUE" if expression.args.get("unique") else ""
clustered = expression.args.get("clustered")
@ -1037,7 +1058,7 @@ class Generator(metaclass=_Generator):
wrapped=False,
)
modifiers = "".join((clustered_sql, replace, unique, postcreate_props_sql))
modifiers = "".join((clustered_sql, replace, refresh, unique, postcreate_props_sql))
postexpression_props_sql = ""
if properties_locs.get(exp.Properties.Location.POST_EXPRESSION):
@ -1096,7 +1117,9 @@ class Generator(metaclass=_Generator):
def describe_sql(self, expression: exp.Describe) -> str:
style = expression.args.get("style")
style = f" {style}" if style else ""
return f"DESCRIBE{style} {self.sql(expression, 'this')}"
partition = self.sql(expression, "partition")
partition = f" {partition}" if partition else ""
return f"DESCRIBE{style} {self.sql(expression, 'this')}{partition}"
def heredoc_sql(self, expression: exp.Heredoc) -> str:
tag = self.sql(expression, "tag")
@ -1195,20 +1218,21 @@ class Generator(metaclass=_Generator):
return f"{this}{specifier}"
def datatype_sql(self, expression: exp.DataType) -> str:
type_value = expression.this
nested = ""
values = ""
interior = self.expressions(expression, flat=True)
type_value = expression.this
if type_value == exp.DataType.Type.USERDEFINED and expression.args.get("kind"):
type_sql = self.sql(expression, "kind")
else:
elif type_value != exp.DataType.Type.NULLABLE or self.SUPPORTS_NULLABLE_TYPES:
type_sql = (
self.TYPE_MAPPING.get(type_value, type_value.value)
if isinstance(type_value, exp.DataType.Type)
else type_value
)
nested = ""
interior = self.expressions(expression, flat=True)
values = ""
else:
return interior
if interior:
if expression.args.get("nested"):
@ -1258,6 +1282,7 @@ class Generator(metaclass=_Generator):
expressions = self.expressions(expression, flat=True)
expressions = f" ({expressions})" if expressions else ""
kind = expression.args["kind"]
kind = self.dialect.INVERSE_CREATABLE_KIND_MAPPING.get(kind) or kind
exists_sql = " IF EXISTS " if expression.args.get("exists") else " "
on_cluster = self.sql(expression, "cluster")
on_cluster = f" {on_cluster}" if on_cluster else ""
@ -1277,7 +1302,7 @@ class Generator(metaclass=_Generator):
def fetch_sql(self, expression: exp.Fetch) -> str:
direction = expression.args.get("direction")
direction = f" {direction}" if direction else ""
count = expression.args.get("count")
count = self.sql(expression, "count")
count = f" {count}" if count else ""
if expression.args.get("percent"):
count = f"{count} PERCENT"
@ -1639,7 +1664,12 @@ class Generator(metaclass=_Generator):
else:
expression_sql = f"{returning}{expression_sql}{on_conflict}"
sql = f"INSERT{hint}{alternative}{ignore}{this}{stored}{by_name}{exists}{where}{expression_sql}"
partition_by = self.sql(expression, "partition")
partition_by = f" {partition_by}" if partition_by else ""
settings = self.sql(expression, "settings")
settings = f" {settings}" if settings else ""
sql = f"INSERT{hint}{alternative}{ignore}{this}{stored}{by_name}{exists}{partition_by}{settings}{where}{expression_sql}"
return self.prepend_ctes(expression, sql)
def intersect_sql(self, expression: exp.Intersect) -> str:
@ -1824,13 +1854,20 @@ class Generator(metaclass=_Generator):
alias = self.sql(expression, "alias")
alias = f" AS {alias}" if alias else ""
direction = self.seg("UNPIVOT" if expression.unpivot else "PIVOT")
field = self.sql(expression, "field")
if field and isinstance(expression.args.get("field"), exp.PivotAny):
field = f"IN ({field})"
include_nulls = expression.args.get("include_nulls")
if include_nulls is not None:
nulls = " INCLUDE NULLS " if include_nulls else " EXCLUDE NULLS "
else:
nulls = ""
return f"{direction}{nulls}({expressions} FOR {field}){alias}"
default_on_null = self.sql(expression, "default_on_null")
default_on_null = f" DEFAULT ON NULL ({default_on_null})" if default_on_null else ""
return f"{direction}{nulls}({expressions} FOR {field}{default_on_null}){alias}"
def version_sql(self, expression: exp.Version) -> str:
this = f"FOR {expression.name}"
@ -2148,15 +2185,7 @@ class Generator(metaclass=_Generator):
this = self.sql(expression, "this")
this = f"{this} " if this else this
siblings = "SIBLINGS " if expression.args.get("siblings") else ""
order = self.op_expressions(f"{this}ORDER {siblings}BY", expression, flat=this or flat) # type: ignore
interpolated_values = [
f"{self.sql(named_expression, 'alias')} AS {self.sql(named_expression, 'this')}"
for named_expression in expression.args.get("interpolate") or []
]
interpolate = (
f" INTERPOLATE ({', '.join(interpolated_values)})" if interpolated_values else ""
)
return f"{order}{interpolate}"
return self.op_expressions(f"{this}ORDER {siblings}BY", expression, flat=this or flat) # type: ignore
def withfill_sql(self, expression: exp.WithFill) -> str:
from_sql = self.sql(expression, "from")
@ -2165,7 +2194,14 @@ class Generator(metaclass=_Generator):
to_sql = f" TO {to_sql}" if to_sql else ""
step_sql = self.sql(expression, "step")
step_sql = f" STEP {step_sql}" if step_sql else ""
return f"WITH FILL{from_sql}{to_sql}{step_sql}"
interpolated_values = [
f"{self.sql(named_expression, 'alias')} AS {self.sql(named_expression, 'this')}"
for named_expression in expression.args.get("interpolate") or []
]
interpolate = (
f" INTERPOLATE ({', '.join(interpolated_values)})" if interpolated_values else ""
)
return f"WITH FILL{from_sql}{to_sql}{step_sql}{interpolate}"
def cluster_sql(self, expression: exp.Cluster) -> str:
return self.op_expressions("CLUSTER BY", expression)
@ -2875,11 +2911,13 @@ class Generator(metaclass=_Generator):
def pivotalias_sql(self, expression: exp.PivotAlias) -> str:
alias = expression.args["alias"]
identifier_alias = isinstance(alias, exp.Identifier)
literal_alias = isinstance(alias, exp.Literal)
if identifier_alias and not self.UNPIVOT_ALIASES_ARE_IDENTIFIERS:
alias.replace(exp.Literal.string(alias.output_name))
elif not identifier_alias and self.UNPIVOT_ALIASES_ARE_IDENTIFIERS:
elif not identifier_alias and literal_alias and self.UNPIVOT_ALIASES_ARE_IDENTIFIERS:
alias.replace(exp.to_identifier(alias.output_name))
return self.alias_sql(expression)
@ -3103,7 +3141,7 @@ class Generator(metaclass=_Generator):
exprs = self.expressions(expression, flat=True)
return f"SET {exprs}"
def altertable_sql(self, expression: exp.AlterTable) -> str:
def alter_sql(self, expression: exp.Alter) -> str:
actions = expression.args["actions"]
if isinstance(actions[0], exp.ColumnDef):
@ -3112,6 +3150,8 @@ class Generator(metaclass=_Generator):
actions = self.expressions(expression, key="actions", prefix="ADD COLUMNS ")
elif isinstance(actions[0], exp.Delete):
actions = self.expressions(expression, key="actions", flat=True)
elif isinstance(actions[0], exp.Query):
actions = "AS " + self.expressions(expression, key="actions")
else:
actions = self.expressions(expression, key="actions", flat=True)
@ -3121,9 +3161,10 @@ class Generator(metaclass=_Generator):
only = " ONLY" if expression.args.get("only") else ""
options = self.expressions(expression, key="options")
options = f", {options}" if options else ""
return f"ALTER TABLE{exists}{only} {self.sql(expression, 'this')}{on_cluster} {actions}{options}"
kind = self.sql(expression, "kind")
return f"ALTER {kind}{exists}{only} {self.sql(expression, 'this')}{on_cluster} {actions}{options}"
def add_column_sql(self, expression: exp.AlterTable) -> str:
def add_column_sql(self, expression: exp.Alter) -> str:
if self.ALTER_TABLE_INCLUDE_COLUMN_KEYWORD:
return self.expressions(
expression,
@ -3312,8 +3353,25 @@ class Generator(metaclass=_Generator):
return f"USE{kind}{this}"
def binary(self, expression: exp.Binary, op: str) -> str:
op = self.maybe_comment(op, comments=expression.comments)
return f"{self.sql(expression, 'this')} {op} {self.sql(expression, 'expression')}"
sqls: t.List[str] = []
stack: t.List[t.Union[str, exp.Expression]] = [expression]
binary_type = type(expression)
while stack:
node = stack.pop()
if type(node) is binary_type:
op_func = node.args.get("operator")
if op_func:
op = f"OPERATOR({self.sql(op_func)})"
stack.append(node.right)
stack.append(f" {self.maybe_comment(op, comments=node.comments)} ")
stack.append(node.left)
else:
sqls.append(self.sql(node))
return "".join(sqls)
def function_fallback_sql(self, expression: exp.Func) -> str:
args = []
@ -3660,9 +3718,6 @@ class Generator(metaclass=_Generator):
table = "" if isinstance(expression.this, exp.Literal) else "TABLE "
return f"REFRESH {table}{this}"
def operator_sql(self, expression: exp.Operator) -> str:
return self.binary(expression, f"OPERATOR({self.sql(expression, 'operator')})")
def toarray_sql(self, expression: exp.ToArray) -> str:
arg = expression.this
if not arg.type:
@ -4041,3 +4096,44 @@ class Generator(metaclass=_Generator):
def summarize_sql(self, expression: exp.Summarize) -> str:
table = " TABLE" if expression.args.get("table") else ""
return f"SUMMARIZE{table} {self.sql(expression.this)}"
def explodinggenerateseries_sql(self, expression: exp.ExplodingGenerateSeries) -> str:
generate_series = exp.GenerateSeries(**expression.args)
parent = expression.parent
if isinstance(parent, (exp.Alias, exp.TableAlias)):
parent = parent.parent
if self.SUPPORTS_EXPLODING_PROJECTIONS and not isinstance(parent, (exp.Table, exp.Unnest)):
return self.sql(exp.Unnest(expressions=[generate_series]))
if isinstance(parent, exp.Select):
self.unsupported("GenerateSeries projection unnesting is not supported.")
return self.sql(generate_series)
def arrayconcat_sql(self, expression: exp.ArrayConcat, name: str = "ARRAY_CONCAT") -> str:
exprs = expression.expressions
if not self.ARRAY_CONCAT_IS_VAR_LEN:
rhs = reduce(lambda x, y: exp.ArrayConcat(this=x, expressions=[y]), exprs)
else:
rhs = self.expressions(expression)
return self.func(name, expression.this, rhs)
def converttimezone_sql(self, expression: exp.ConvertTimezone) -> str:
if self.SUPPORTS_CONVERT_TIMEZONE:
return self.function_fallback_sql(expression)
source_tz = expression.args.get("source_tz")
target_tz = expression.args.get("target_tz")
timestamp = expression.args.get("timestamp")
if source_tz and timestamp:
timestamp = exp.AtTimeZone(
this=exp.cast(timestamp, exp.DataType.Type.TIMESTAMPNTZ), zone=source_tz
)
expr = exp.AtTimeZone(this=timestamp, zone=target_tz)
return self.sql(expr)

View file

@ -128,6 +128,13 @@ class _TypeAnnotator(type):
klass.COERCES_TO[data_type] = coerces_to.copy()
coerces_to |= {data_type}
# NULL can be coerced to any type, so e.g. NULL + 1 will have type INT
klass.COERCES_TO[exp.DataType.Type.NULL] = {
*text_precedence,
*numeric_precedence,
*timelike_precedence,
}
return klass
@ -201,31 +208,47 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
for name, source in scope.sources.items():
if not isinstance(source, Scope):
continue
if isinstance(source.expression, exp.UDTF):
expression = source.expression
if isinstance(expression, exp.UDTF):
values = []
if isinstance(source.expression, exp.Lateral):
if isinstance(source.expression.this, exp.Explode):
values = [source.expression.this.this]
elif isinstance(source.expression, exp.Unnest):
values = [source.expression]
if isinstance(expression, exp.Lateral):
if isinstance(expression.this, exp.Explode):
values = [expression.this.this]
elif isinstance(expression, exp.Unnest):
values = [expression]
else:
values = source.expression.expressions[0].expressions
values = expression.expressions[0].expressions
if not values:
continue
selects[name] = {
alias: column
for alias, column in zip(
source.expression.alias_column_names,
values,
alias: column.type
for alias, column in zip(expression.alias_column_names, values)
}
elif isinstance(expression, exp.SetOperation) and len(expression.left.selects) == len(
expression.right.selects
):
if expression.args.get("by_name"):
r_type_by_select = {s.alias_or_name: s.type for s in expression.right.selects}
selects[name] = {
s.alias_or_name: self._maybe_coerce(
t.cast(exp.DataType, s.type),
r_type_by_select.get(s.alias_or_name) or exp.DataType.Type.UNKNOWN,
)
for s in expression.left.selects
}
else:
selects[name] = {
select.alias_or_name: select for select in source.expression.selects
ls.alias_or_name: self._maybe_coerce(
t.cast(exp.DataType, ls.type), t.cast(exp.DataType, rs.type)
)
for ls, rs in zip(expression.left.selects, expression.right.selects)
}
else:
selects[name] = {s.alias_or_name: s.type for s in expression.selects}
# First annotate the current scope's column references
for col in scope.columns:
@ -237,7 +260,7 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
self._set_type(col, self.schema.get_column_type(source, col))
elif source:
if col.table in selects and col.name in selects[col.table]:
self._set_type(col, selects[col.table][col.name].type)
self._set_type(col, selects[col.table][col.name])
elif isinstance(source.expression, exp.Unnest):
self._set_type(col, source.expression.type)
@ -264,15 +287,13 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
def _maybe_coerce(
self, type1: exp.DataType | exp.DataType.Type, type2: exp.DataType | exp.DataType.Type
) -> exp.DataType | exp.DataType.Type:
) -> exp.DataType:
type1_value = type1.this if isinstance(type1, exp.DataType) else type1
type2_value = type2.this if isinstance(type2, exp.DataType) else type2
# We propagate the NULL / UNKNOWN types upwards if found
if exp.DataType.Type.NULL in (type1_value, type2_value):
return exp.DataType.Type.NULL
# We propagate the UNKNOWN type upwards if found
if exp.DataType.Type.UNKNOWN in (type1_value, type2_value):
return exp.DataType.Type.UNKNOWN
return exp.DataType.build("unknown")
return type2_value if type2_value in self.coerces_to.get(type1_value, {}) else type1_value
@ -282,17 +303,7 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
left, right = expression.left, expression.right
left_type, right_type = left.type.this, right.type.this # type: ignore
if isinstance(expression, exp.Connector):
if left_type == exp.DataType.Type.NULL and right_type == exp.DataType.Type.NULL:
self._set_type(expression, exp.DataType.Type.NULL)
elif exp.DataType.Type.NULL in (left_type, right_type):
self._set_type(
expression,
exp.DataType.build("NULLABLE", expressions=exp.DataType.build("BOOLEAN")),
)
else:
self._set_type(expression, exp.DataType.Type.BOOLEAN)
elif isinstance(expression, exp.Predicate):
if isinstance(expression, (exp.Connector, exp.Predicate)):
self._set_type(expression, exp.DataType.Type.BOOLEAN)
elif (left_type, right_type) in self.binary_coercions:
self._set_type(expression, self.binary_coercions[(left_type, right_type)](left, right))
@ -351,7 +362,7 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
last_datatype = expr_type
break
if not expr_type.is_type(exp.DataType.Type.NULL, exp.DataType.Type.UNKNOWN):
if not expr_type.is_type(exp.DataType.Type.UNKNOWN):
last_datatype = self._maybe_coerce(last_datatype or expr_type, expr_type)
self._set_type(expression, last_datatype or exp.DataType.Type.UNKNOWN)

View file

@ -40,6 +40,8 @@ def replace_date_funcs(node: exp.Expression) -> exp.Expression:
isinstance(node, (exp.Date, exp.TsOrDsToDate))
and not node.expressions
and not node.args.get("zone")
and node.this.is_string
and is_iso_date(node.this.name)
):
return exp.cast(node.this, to=exp.DataType.Type.DATE)
if isinstance(node, exp.Timestamp) and not node.args.get("zone"):
@ -90,6 +92,12 @@ def remove_redundant_casts(expression: exp.Expression) -> exp.Expression:
and expression.to.this == expression.this.type.this
):
return expression.this
if (
isinstance(expression, (exp.Date, exp.TsOrDsToDate))
and expression.this.type
and expression.this.type.this == exp.DataType.Type.DATE
):
return expression.this
return expression

View file

@ -19,8 +19,12 @@ def normalize_identifiers(expression: str, dialect: DialectType = None) -> exp.I
def normalize_identifiers(expression, dialect=None):
"""
Normalize all unquoted identifiers to either lower or upper case, depending
on the dialect. This essentially makes those identifiers case-insensitive.
Normalize identifiers by converting them to either lower or upper case,
ensuring the semantics are preserved in each case (e.g. by respecting
case-sensitivity).
This transformation reflects how identifiers would be resolved by the engine corresponding
to each SQL dialect, and plays a very important role in the standardization of the AST.
It's possible to make this a no-op by adding a special comment next to the
identifier of interest:
@ -30,7 +34,7 @@ def normalize_identifiers(expression, dialect=None):
In this example, the identifier `a` will not be normalized.
Note:
Some dialects (e.g. BigQuery) treat identifiers as case-insensitive even
Some dialects (e.g. DuckDB) treat all identifiers as case-insensitive even
when they're quoted, so in these cases all identifiers are normalized.
Example:

View file

@ -30,6 +30,7 @@ def qualify(
validate_qualify_columns: bool = True,
quote_identifiers: bool = True,
identify: bool = True,
infer_csv_schemas: bool = False,
) -> exp.Expression:
"""
Rewrite sqlglot AST to have normalized and qualified tables and columns.
@ -60,13 +61,21 @@ def qualify(
This step is necessary to ensure correctness for case sensitive queries.
But this flag is provided in case this step is performed at a later time.
identify: If True, quote all identifiers, else only necessary ones.
infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs' schemas.
Returns:
The qualified expression.
"""
schema = ensure_schema(schema, dialect=dialect)
expression = normalize_identifiers(expression, dialect=dialect)
expression = qualify_tables(expression, db=db, catalog=catalog, schema=schema, dialect=dialect)
expression = qualify_tables(
expression,
db=db,
catalog=catalog,
schema=schema,
dialect=dialect,
infer_csv_schemas=infer_csv_schemas,
)
if isolate_tables:
expression = isolate_table_selects(expression, schema=schema)

View file

@ -275,6 +275,17 @@ def _expand_alias_refs(scope: Scope, resolver: Resolver, expand_only_groupby: bo
if isinstance(projection, exp.Alias):
alias_to_expression[projection.alias] = (projection.this, i + 1)
parent_scope = scope
while parent_scope.is_union:
parent_scope = parent_scope.parent
# We shouldn't expand aliases if they match the recursive CTE's columns
if parent_scope.is_cte:
cte = parent_scope.expression.parent
if cte.find_ancestor(exp.With).recursive:
for recursive_cte_column in cte.args["alias"].columns or cte.this.selects:
alias_to_expression.pop(recursive_cte_column.output_name, None)
replace_columns(expression.args.get("where"))
replace_columns(expression.args.get("group"), literal_index=True)
replace_columns(expression.args.get("having"), resolve_table=True)

View file

@ -18,6 +18,7 @@ def qualify_tables(
db: t.Optional[str | exp.Identifier] = None,
catalog: t.Optional[str | exp.Identifier] = None,
schema: t.Optional[Schema] = None,
infer_csv_schemas: bool = False,
dialect: DialectType = None,
) -> E:
"""
@ -39,6 +40,7 @@ def qualify_tables(
db: Database name
catalog: Catalog name
schema: A schema to populate
infer_csv_schemas: Whether to scan READ_CSV calls in order to infer the CSVs' schemas.
dialect: The dialect to parse catalog and schema into.
Returns:
@ -102,7 +104,7 @@ def qualify_tables(
"alias", exp.TableAlias(this=exp.to_identifier(next_alias_name()))
)
if schema and isinstance(source.this, exp.ReadCSV):
if infer_csv_schemas and schema and isinstance(source.this, exp.ReadCSV):
with csv_reader(source.this) as reader:
header = next(reader)
columns = next(reader)

View file

@ -65,6 +65,7 @@ class Scope:
scope_type=ScopeType.ROOT,
lateral_sources=None,
cte_sources=None,
can_be_correlated=None,
):
self.expression = expression
self.sources = sources or {}
@ -81,6 +82,7 @@ class Scope:
self.cte_scopes = []
self.union_scopes = []
self.udtf_scopes = []
self.can_be_correlated = can_be_correlated
self.clear_cache()
def clear_cache(self):
@ -110,6 +112,8 @@ class Scope:
scope_type=scope_type,
cte_sources={**self.cte_sources, **(cte_sources or {})},
lateral_sources=lateral_sources.copy() if lateral_sources else None,
can_be_correlated=self.can_be_correlated
or scope_type in (ScopeType.SUBQUERY, ScopeType.UDTF),
**kwargs,
)
@ -261,7 +265,11 @@ class Scope:
external_columns = [
column
for scope in itertools.chain(self.subquery_scopes, self.udtf_scopes)
for scope in itertools.chain(
self.subquery_scopes,
self.udtf_scopes,
(dts for dts in self.derived_table_scopes if dts.can_be_correlated),
)
for column in scope.external_columns
]
@ -425,10 +433,7 @@ class Scope:
@property
def is_correlated_subquery(self):
"""Determine if this scope is a correlated subquery"""
return bool(
(self.is_subquery or (self.parent and isinstance(self.parent.expression, exp.Lateral)))
and self.external_columns
)
return bool(self.can_be_correlated and self.external_columns)
def rename_source(self, old_name, new_name):
"""Rename a source in this scope"""

View file

@ -117,6 +117,29 @@ def build_pad(args: t.List, is_left: bool = True):
)
def build_array_constructor(
exp_class: t.Type[E], args: t.List, bracket_kind: TokenType, dialect: Dialect
) -> exp.Expression:
array_exp = exp_class(expressions=args)
if exp_class == exp.Array and dialect.HAS_DISTINCT_ARRAY_CONSTRUCTORS:
array_exp.set("bracket_notation", bracket_kind == TokenType.L_BRACKET)
return array_exp
def build_convert_timezone(
args: t.List, default_source_tz: t.Optional[str] = None
) -> t.Union[exp.ConvertTimezone, exp.Anonymous]:
if len(args) == 2:
source_tz = exp.Literal.string(default_source_tz) if default_source_tz else None
return exp.ConvertTimezone(
source_tz=source_tz, target_tz=seq_get(args, 0), timestamp=seq_get(args, 1)
)
return exp.ConvertTimezone.from_arg_list(args)
class _Parser(type):
def __new__(cls, clsname, bases, attrs):
klass = super().__new__(cls, clsname, bases, attrs)
@ -144,6 +167,7 @@ class Parser(metaclass=_Parser):
FUNCTIONS: t.Dict[str, t.Callable] = {
**{name: func.from_arg_list for name, func in exp.FUNCTION_BY_NAME.items()},
"ARRAY": lambda args, dialect: exp.Array(expressions=args),
"CONCAT": lambda args, dialect: exp.Concat(
expressions=args,
safe=not dialect.STRICT_STRING_CONCAT,
@ -154,10 +178,16 @@ class Parser(metaclass=_Parser):
safe=not dialect.STRICT_STRING_CONCAT,
coalesce=dialect.CONCAT_COALESCE,
),
"CONVERT_TIMEZONE": build_convert_timezone,
"DATE_TO_DATE_STR": lambda args: exp.Cast(
this=seq_get(args, 0),
to=exp.DataType(this=exp.DataType.Type.TEXT),
),
"GENERATE_DATE_ARRAY": lambda args: exp.GenerateDateArray(
start=seq_get(args, 0),
end=seq_get(args, 1),
step=seq_get(args, 2) or exp.Interval(this=exp.Literal.number(1), unit=exp.var("DAY")),
),
"GLOB": lambda args: exp.Glob(this=seq_get(args, 1), expression=seq_get(args, 0)),
"HEX": build_hex,
"JSON_EXTRACT": build_extract_json_with_path(exp.JSONExtract),
@ -192,6 +222,7 @@ class Parser(metaclass=_Parser):
"UNNEST": lambda args: exp.Unnest(expressions=ensure_list(seq_get(args, 0))),
"UPPER": build_upper,
"VAR_MAP": build_var_map,
"COALESCE": lambda args: exp.Coalesce(this=seq_get(args, 0), expressions=args[1:]),
}
NO_PAREN_FUNCTIONS = {
@ -374,6 +405,11 @@ class Parser(metaclass=_Parser):
*DB_CREATABLES,
}
ALTERABLES = {
TokenType.TABLE,
TokenType.VIEW,
}
# Tokens that can represent identifiers
ID_VAR_TOKENS = {
TokenType.ALL,
@ -433,6 +469,7 @@ class Parser(metaclass=_Parser):
TokenType.RECURSIVE,
TokenType.REFERENCES,
TokenType.REFRESH,
TokenType.RENAME,
TokenType.REPLACE,
TokenType.RIGHT,
TokenType.ROLLUP,
@ -842,6 +879,7 @@ class Parser(metaclass=_Parser):
"DYNAMIC": lambda self: self.expression(exp.DynamicProperty),
"DISTKEY": lambda self: self._parse_distkey(),
"DISTSTYLE": lambda self: self._parse_property_assignment(exp.DistStyleProperty),
"EMPTY": lambda self: self.expression(exp.EmptyProperty),
"ENGINE": lambda self: self._parse_property_assignment(exp.EngineProperty),
"EXECUTE": lambda self: self._parse_property_assignment(exp.ExecuteAsProperty),
"EXTERNAL": lambda self: self.expression(exp.ExternalProperty),
@ -885,6 +923,7 @@ class Parser(metaclass=_Parser):
"REMOTE": lambda self: self._parse_remote_with_connection(),
"RETURNS": lambda self: self._parse_returns(),
"STRICT": lambda self: self.expression(exp.StrictProperty),
"STREAMING": lambda self: self.expression(exp.StreamingTableProperty),
"ROW": lambda self: self._parse_row(),
"ROW_FORMAT": lambda self: self._parse_property_assignment(exp.RowFormatProperty),
"SAMPLE": lambda self: self.expression(
@ -892,9 +931,7 @@ class Parser(metaclass=_Parser):
),
"SECURE": lambda self: self.expression(exp.SecureProperty),
"SET": lambda self: self.expression(exp.SetProperty, multi=False),
"SETTINGS": lambda self: self.expression(
exp.SettingsProperty, expressions=self._parse_csv(self._parse_set_item)
),
"SETTINGS": lambda self: self._parse_settings_property(),
"SHARING": lambda self: self._parse_property_assignment(exp.SharingProperty),
"SORTKEY": lambda self: self._parse_sortkey(),
"SOURCE": lambda self: self._parse_dict_property(this="SOURCE"),
@ -992,6 +1029,7 @@ class Parser(metaclass=_Parser):
"DROP": lambda self: self._parse_alter_table_drop(),
"RENAME": lambda self: self._parse_alter_table_rename(),
"SET": lambda self: self._parse_alter_table_set(),
"AS": lambda self: self._parse_select(),
}
ALTER_ALTER_PARSERS = {
@ -1628,7 +1666,7 @@ class Parser(metaclass=_Parser):
temporary = self._match(TokenType.TEMPORARY)
materialized = self._match_text_seq("MATERIALIZED")
kind = self._match_set(self.CREATABLES) and self._prev.text
kind = self._match_set(self.CREATABLES) and self._prev.text.upper()
if not kind:
return self._parse_as_command(start)
@ -1650,7 +1688,7 @@ class Parser(metaclass=_Parser):
exists=if_exists,
this=table,
expressions=expressions,
kind=kind.upper(),
kind=self.dialect.CREATABLE_KIND_MAPPING.get(kind) or kind,
temporary=temporary,
materialized=materialized,
cascade=self._match_text_seq("CASCADE"),
@ -1676,6 +1714,7 @@ class Parser(metaclass=_Parser):
or self._match_pair(TokenType.OR, TokenType.REPLACE)
or self._match_pair(TokenType.OR, TokenType.ALTER)
)
refresh = self._match_pair(TokenType.OR, TokenType.REFRESH)
unique = self._match(TokenType.UNIQUE)
@ -1792,7 +1831,6 @@ class Parser(metaclass=_Parser):
# exp.Properties.Location.POST_INDEX
extend_props(self._parse_properties())
if not index:
break
else:
@ -1813,12 +1851,14 @@ class Parser(metaclass=_Parser):
if self._curr and not self._match_set((TokenType.R_PAREN, TokenType.COMMA), advance=False):
return self._parse_as_command(start)
create_kind_text = create_token.text.upper()
return self.expression(
exp.Create,
comments=comments,
this=this,
kind=create_token.text.upper(),
kind=self.dialect.CREATABLE_KIND_MAPPING.get(create_kind_text) or create_kind_text,
replace=replace,
refresh=refresh,
unique=unique,
expression=expression,
exists=exists,
@ -1979,6 +2019,11 @@ class Parser(metaclass=_Parser):
exp.FallbackProperty, no=no, protection=self._match_text_seq("PROTECTION")
)
def _parse_settings_property(self) -> exp.SettingsProperty:
return self.expression(
exp.SettingsProperty, expressions=self._parse_csv(self._parse_assignment)
)
def _parse_volatile_property(self) -> exp.VolatileProperty | exp.StabilityProperty:
if self._index >= 2:
pre_volatile_token = self._tokens[self._index - 2]
@ -2451,8 +2496,14 @@ class Parser(metaclass=_Parser):
this = self._parse_table(schema=True)
properties = self._parse_properties()
expressions = properties.expressions if properties else None
partition = self._parse_partition()
return self.expression(
exp.Describe, this=this, style=style, kind=kind, expressions=expressions
exp.Describe,
this=this,
style=style,
kind=kind,
expressions=expressions,
partition=partition,
)
def _parse_insert(self) -> exp.Insert:
@ -2498,6 +2549,8 @@ class Parser(metaclass=_Parser):
by_name=self._match_text_seq("BY", "NAME"),
exists=self._parse_exists(),
where=self._match_pair(TokenType.REPLACE, TokenType.WHERE) and self._parse_assignment(),
partition=self._match(TokenType.PARTITION_BY) and self._parse_partitioned_by(),
settings=self._match_text_seq("SETTINGS") and self._parse_settings_property(),
expression=self._parse_derived_table_values() or self._parse_ddl_select(),
conflict=self._parse_on_conflict(),
returning=returning or self._parse_returning(),
@ -2830,6 +2883,10 @@ class Parser(metaclass=_Parser):
table = self._match(TokenType.TABLE)
this = self._parse_select() or self._parse_string() or self._parse_table()
return self.expression(exp.Summarize, this=this, table=table)
elif self._match(TokenType.DESCRIBE):
this = self._parse_describe()
elif self._match_text_seq("STREAM"):
this = self.expression(exp.Stream, this=self._parse_function())
else:
this = None
@ -3173,6 +3230,15 @@ class Parser(metaclass=_Parser):
self._match_set(self.JOIN_KINDS) and self._prev,
)
def _parse_using_identifiers(self) -> t.List[exp.Expression]:
def _parse_column_as_identifier() -> t.Optional[exp.Expression]:
this = self._parse_column()
if isinstance(this, exp.Column):
return this.this
return this
return self._parse_wrapped_csv(_parse_column_as_identifier, optional=True)
def _parse_join(
self, skip_join_token: bool = False, parse_bracket: bool = False
) -> t.Optional[exp.Join]:
@ -3213,9 +3279,11 @@ class Parser(metaclass=_Parser):
if self._match(TokenType.ON):
kwargs["on"] = self._parse_assignment()
elif self._match(TokenType.USING):
kwargs["using"] = self._parse_wrapped_id_vars()
elif not isinstance(kwargs["this"], exp.Unnest) and not (
kind and kind.token_type == TokenType.CROSS
kwargs["using"] = self._parse_using_identifiers()
elif (
not (outer_apply or cross_apply)
and not isinstance(kwargs["this"], exp.Unnest)
and not (kind and kind.token_type == TokenType.CROSS)
):
index = self._index
joins: t.Optional[list] = list(self._parse_joins())
@ -3223,7 +3291,7 @@ class Parser(metaclass=_Parser):
if joins and self._match(TokenType.ON):
kwargs["on"] = self._parse_assignment()
elif joins and self._match(TokenType.USING):
kwargs["using"] = self._parse_wrapped_id_vars()
kwargs["using"] = self._parse_using_identifiers()
else:
joins = None
self._retreat(index)
@ -3607,7 +3675,10 @@ class Parser(metaclass=_Parser):
def _parse_derived_table_values(self) -> t.Optional[exp.Values]:
is_derived = self._match_pair(TokenType.L_PAREN, TokenType.VALUES)
if not is_derived and not self._match_text_seq("VALUES"):
if not is_derived and not (
# ClickHouse's `FORMAT Values` is equivalent to `VALUES`
self._match_text_seq("VALUES") or self._match_text_seq("FORMAT", "VALUES")
):
return None
expressions = self._parse_csv(self._parse_value)
@ -3707,13 +3778,15 @@ class Parser(metaclass=_Parser):
exp.Pivot, this=this, expressions=expressions, using=using, group=group
)
def _parse_pivot_in(self) -> exp.In:
def _parse_pivot_in(self) -> exp.In | exp.PivotAny:
def _parse_aliased_expression() -> t.Optional[exp.Expression]:
this = self._parse_assignment()
this = self._parse_select_or_expression()
self._match(TokenType.ALIAS)
alias = self._parse_field()
alias = self._parse_bitwise()
if alias:
if isinstance(alias, exp.Column) and not alias.db:
alias = alias.this
return self.expression(exp.PivotAlias, this=this, alias=alias)
return this
@ -3723,10 +3796,14 @@ class Parser(metaclass=_Parser):
if not self._match_pair(TokenType.IN, TokenType.L_PAREN):
self.raise_error("Expecting IN (")
if self._match(TokenType.ANY):
expr: exp.PivotAny | exp.In = self.expression(exp.PivotAny, this=self._parse_order())
else:
aliased_expressions = self._parse_csv(_parse_aliased_expression)
expr = self.expression(exp.In, this=value, expressions=aliased_expressions)
self._match_r_paren()
return self.expression(exp.In, this=value, expressions=aliased_expressions)
return expr
def _parse_pivot(self) -> t.Optional[exp.Pivot]:
index = self._index
@ -3763,6 +3840,9 @@ class Parser(metaclass=_Parser):
self.raise_error("Expecting FOR")
field = self._parse_pivot_in()
default_on_null = self._match_text_seq("DEFAULT", "ON", "NULL") and self._parse_wrapped(
self._parse_bitwise
)
self._match_r_paren()
@ -3772,6 +3852,7 @@ class Parser(metaclass=_Parser):
field=field,
unpivot=unpivot,
include_nulls=include_nulls,
default_on_null=default_on_null,
)
if not self._match_set((TokenType.PIVOT, TokenType.UNPIVOT), advance=False):
@ -3934,7 +4015,6 @@ class Parser(metaclass=_Parser):
exp.Order,
this=this,
expressions=self._parse_csv(self._parse_ordered),
interpolate=self._parse_interpolate(),
siblings=siblings,
)
@ -3979,6 +4059,7 @@ class Parser(metaclass=_Parser):
"from": self._match(TokenType.FROM) and self._parse_bitwise(),
"to": self._match_text_seq("TO") and self._parse_bitwise(),
"step": self._match_text_seq("STEP") and self._parse_bitwise(),
"interpolate": self._parse_interpolate(),
},
)
else:
@ -4132,6 +4213,11 @@ class Parser(metaclass=_Parser):
def _parse_assignment(self) -> t.Optional[exp.Expression]:
this = self._parse_disjunction()
if not this and self._next and self._next.token_type in self.ASSIGNMENT:
# This allows us to parse <non-identifier token> := <expr>
this = exp.column(
t.cast(str, self._advance_any(ignore_reserved=True) and self._prev.text)
)
while self._match_set(self.ASSIGNMENT):
this = self.expression(
@ -4175,13 +4261,19 @@ class Parser(metaclass=_Parser):
this = self.expression(exp.Not, this=this)
if negate:
this = self.expression(exp.Not, this=this)
this = self._negate_range(this)
if self._match(TokenType.IS):
this = self._parse_is(this)
return this
def _negate_range(self, this: t.Optional[exp.Expression] = None) -> t.Optional[exp.Expression]:
if not this:
return this
return self.expression(exp.Not, this=this)
def _parse_is(self, this: t.Optional[exp.Expression]) -> t.Optional[exp.Expression]:
index = self._index - 1
negate = self._match(TokenType.NOT)
@ -4322,7 +4414,26 @@ class Parser(metaclass=_Parser):
return this
def _parse_term(self) -> t.Optional[exp.Expression]:
return self._parse_tokens(self._parse_factor, self.TERM)
this = self._parse_factor()
while self._match_set(self.TERM):
klass = self.TERM[self._prev.token_type]
comments = self._prev_comments
expression = self._parse_factor()
this = self.expression(klass, this=this, comments=comments, expression=expression)
if isinstance(this, exp.Collate):
expr = this.expression
# Preserve collations such as pg_catalog."default" (Postgres) as columns, otherwise
# fallback to Identifier / Var
if isinstance(expr, exp.Column) and len(expr.parts) == 1:
ident = expr.this
if isinstance(ident, exp.Identifier):
this.set("expression", ident if ident.quoted else exp.var(ident.name))
return this
def _parse_factor(self) -> t.Optional[exp.Expression]:
parse_method = self._parse_exponent if self.EXPONENT else self._parse_unary
@ -4610,6 +4721,7 @@ class Parser(metaclass=_Parser):
matched_array = self._match(TokenType.ARRAY)
while self._curr:
datatype_token = self._prev.token_type
matched_l_bracket = self._match(TokenType.L_BRACKET)
if not matched_l_bracket and not matched_array:
break
@ -4619,8 +4731,12 @@ class Parser(metaclass=_Parser):
if (
values
and not schema
and this.is_type(exp.DataType.Type.ARRAY, exp.DataType.Type.MAP)
and (
not self.dialect.SUPPORTS_FIXED_SIZE_ARRAYS or datatype_token == TokenType.ARRAY
)
):
# Retreating here means that we should not parse the following values as part of the data type, e.g. in DuckDB
# ARRAY[1] should retreat and instead be parsed into exp.Array in contrast to INT[x][y] which denotes a fixed-size array data type
self._retreat(index)
break
@ -5407,11 +5523,18 @@ class Parser(metaclass=_Parser):
if bracket_kind == TokenType.L_BRACE:
this = self.expression(exp.Struct, expressions=self._kv_to_prop_eq(expressions))
elif not this:
this = self.expression(exp.Array, expressions=expressions)
this = build_array_constructor(
exp.Array, args=expressions, bracket_kind=bracket_kind, dialect=self.dialect
)
else:
constructor_type = self.ARRAY_CONSTRUCTORS.get(this.name.upper())
if constructor_type:
return self.expression(constructor_type, expressions=expressions)
return build_array_constructor(
constructor_type,
args=expressions,
bracket_kind=bracket_kind,
dialect=self.dialect,
)
expressions = apply_index_offset(this, expressions, -self.dialect.INDEX_OFFSET)
this = self.expression(exp.Bracket, this=this, expressions=expressions)
@ -6440,10 +6563,11 @@ class Parser(metaclass=_Parser):
return alter_set
def _parse_alter(self) -> exp.AlterTable | exp.Command:
def _parse_alter(self) -> exp.Alter | exp.Command:
start = self._prev
if not self._match(TokenType.TABLE):
alter_token = self._match_set(self.ALTERABLES) and self._prev
if not alter_token:
return self._parse_as_command(start)
exists = self._parse_exists()
@ -6461,8 +6585,9 @@ class Parser(metaclass=_Parser):
if not self._curr and actions:
return self.expression(
exp.AlterTable,
exp.Alter,
this=this,
kind=alter_token.text.upper(),
exists=exists,
actions=actions,
only=only,

View file

@ -340,6 +340,7 @@ class TokenType(AutoName):
RANGE = auto()
RECURSIVE = auto()
REFRESH = auto()
RENAME = auto()
REPLACE = auto()
RETURNING = auto()
REFERENCES = auto()
@ -529,6 +530,7 @@ class _Tokenizer(type):
},
heredoc_tag_is_identifier=klass.HEREDOC_TAG_IS_IDENTIFIER,
string_escapes_allowed_in_raw_strings=klass.STRING_ESCAPES_ALLOWED_IN_RAW_STRINGS,
nested_comments=klass.NESTED_COMMENTS,
)
token_types = RsTokenTypeSettings(
bit_string=_TOKEN_TYPE_TO_INDEX[TokenType.BIT_STRING],
@ -608,6 +610,8 @@ class Tokenizer(metaclass=_Tokenizer):
# Whether string escape characters function as such when placed within raw strings
STRING_ESCAPES_ALLOWED_IN_RAW_STRINGS = True
NESTED_COMMENTS = True
# Autofilled
_COMMENTS: t.Dict[str, str] = {}
_FORMAT_STRINGS: t.Dict[str, t.Tuple[str, TokenType]] = {}
@ -753,6 +757,7 @@ class Tokenizer(metaclass=_Tokenizer):
"RANGE": TokenType.RANGE,
"RECURSIVE": TokenType.RECURSIVE,
"REGEXP": TokenType.RLIKE,
"RENAME": TokenType.RENAME,
"REPLACE": TokenType.REPLACE,
"RETURNING": TokenType.RETURNING,
"REFERENCES": TokenType.REFERENCES,
@ -913,6 +918,7 @@ class Tokenizer(metaclass=_Tokenizer):
TokenType.EXECUTE,
TokenType.FETCH,
TokenType.SHOW,
TokenType.RENAME,
}
COMMAND_PREFIX_TOKENS = {TokenType.SEMICOLON, TokenType.BEGIN}
@ -1181,7 +1187,11 @@ class Tokenizer(metaclass=_Tokenizer):
self._advance(alnum=True)
# Nested comments are allowed by some dialects, e.g. databricks, duckdb, postgres
if not self._end and self._chars(comment_end_size) == comment_start:
if (
self.NESTED_COMMENTS
and not self._end
and self._chars(comment_end_size) == comment_start
):
self._advance(comment_start_size)
comment_count += 1

View file

@ -55,6 +55,76 @@ def preprocess(
return _to_sql
def unnest_generate_date_array_using_recursive_cte(expression: exp.Expression) -> exp.Expression:
if isinstance(expression, exp.Select):
count = 0
recursive_ctes = []
for unnest in expression.find_all(exp.Unnest):
if (
not isinstance(unnest.parent, (exp.From, exp.Join))
or len(unnest.expressions) != 1
or not isinstance(unnest.expressions[0], exp.GenerateDateArray)
):
continue
generate_date_array = unnest.expressions[0]
start = generate_date_array.args.get("start")
end = generate_date_array.args.get("end")
step = generate_date_array.args.get("step")
if not start or not end or not isinstance(step, exp.Interval):
continue
alias = unnest.args.get("alias")
column_name = alias.columns[0] if isinstance(alias, exp.TableAlias) else "date_value"
start = exp.cast(start, "date")
date_add = exp.func(
"date_add", column_name, exp.Literal.number(step.name), step.args.get("unit")
)
cast_date_add = exp.cast(date_add, "date")
cte_name = "_generated_dates" + (f"_{count}" if count else "")
base_query = exp.select(start.as_(column_name))
recursive_query = (
exp.select(cast_date_add)
.from_(cte_name)
.where(cast_date_add <= exp.cast(end, "date"))
)
cte_query = base_query.union(recursive_query, distinct=False)
generate_dates_query = exp.select(column_name).from_(cte_name)
unnest.replace(generate_dates_query.subquery(cte_name))
recursive_ctes.append(
exp.alias_(exp.CTE(this=cte_query), cte_name, table=[column_name])
)
count += 1
if recursive_ctes:
with_expression = expression.args.get("with") or exp.With()
with_expression.set("recursive", True)
with_expression.set("expressions", [*recursive_ctes, *with_expression.expressions])
expression.set("with", with_expression)
return expression
def unnest_generate_series(expression: exp.Expression) -> exp.Expression:
"""Unnests GENERATE_SERIES or SEQUENCE table references."""
this = expression.this
if isinstance(expression, exp.Table) and isinstance(this, exp.GenerateSeries):
unnest = exp.Unnest(expressions=[this])
if expression.alias:
return exp.alias_(unnest, alias="_u", table=[expression.alias], copy=False)
return unnest
return expression
def unalias_group(expression: exp.Expression) -> exp.Expression:
"""
Replace references to select aliases in GROUP BY clauses.

2
sqlglotrs/Cargo.lock generated
View file

@ -188,7 +188,7 @@ checksum = "4dccd0940a2dcdf68d092b8cbab7dc0ad8fa938bf95787e1b916b0e3d0e8e970"
[[package]]
name = "sqlglotrs"
version = "0.2.8"
version = "0.2.9"
dependencies = [
"pyo3",
]

View file

@ -1,6 +1,6 @@
[package]
name = "sqlglotrs"
version = "0.2.8"
version = "0.2.9"
edition = "2021"
[lib]

View file

@ -77,6 +77,7 @@ pub struct TokenizerSettings {
pub command_prefix_tokens: HashSet<TokenType>,
pub heredoc_tag_is_identifier: bool,
pub string_escapes_allowed_in_raw_strings: bool,
pub nested_comments: bool,
}
#[pymethods]
@ -100,6 +101,7 @@ impl TokenizerSettings {
command_prefix_tokens: HashSet<TokenType>,
heredoc_tag_is_identifier: bool,
string_escapes_allowed_in_raw_strings: bool,
nested_comments: bool,
) -> Self {
let to_char = |v: &String| {
if v.len() == 1 {
@ -150,6 +152,7 @@ impl TokenizerSettings {
command_prefix_tokens,
heredoc_tag_is_identifier,
string_escapes_allowed_in_raw_strings,
nested_comments,
}
}
}

View file

@ -375,7 +375,7 @@ impl<'a> TokenizerState<'a> {
self.advance(1)?;
// Nested comments are allowed by some dialects, e.g. databricks, duckdb, postgres
if !self.is_end && self.chars(comment_start_size) == *comment_start {
if self.settings.nested_comments && !self.is_end && self.chars(comment_start_size) == *comment_start {
self.advance(comment_start_size as isize)?;
comment_count += 1
}

View file

@ -6,6 +6,15 @@ class TestAthena(Validator):
maxDiff = None
def test_athena(self):
self.validate_identity(
"CREATE TABLE IF NOT EXISTS t (name STRING) LOCATION 's3://bucket/tmp/mytable/' TBLPROPERTIES ('table_type'='iceberg', 'FORMAT'='parquet')"
)
self.validate_identity(
"UNLOAD (SELECT name1, address1, comment1, key1 FROM table1) "
"TO 's3://amzn-s3-demo-bucket/ partitioned/' "
"WITH (format = 'TEXTFILE', partitioned_by = ARRAY['key1'])",
check_command_warning=True,
)
self.validate_identity(
"""USING EXTERNAL FUNCTION some_function(input VARBINARY)
RETURNS VARCHAR
@ -14,7 +23,3 @@ class TestAthena(Validator):
some_function(1)""",
check_command_warning=True,
)
self.validate_identity(
"CREATE TABLE IF NOT EXISTS t (name STRING) LOCATION 's3://bucket/tmp/mytable/' TBLPROPERTIES ('table_type'='iceberg', 'FORMAT'='parquet')"
)

View file

@ -1,4 +1,6 @@
from unittest import mock
import datetime
import pytz
from sqlglot import (
ErrorLevel,
@ -103,6 +105,7 @@ LANGUAGE js AS
select_with_quoted_udf = self.validate_identity("SELECT `p.d.UdF`(data) FROM `p.d.t`")
self.assertEqual(select_with_quoted_udf.selects[0].name, "p.d.UdF")
self.validate_identity("SELECT * FROM READ_CSV('bla.csv')")
self.validate_identity("CAST(x AS STRUCT<list ARRAY<INT64>>)")
self.validate_identity("assert.true(1 = 1)")
self.validate_identity("SELECT ARRAY_TO_STRING(list, '--') AS text")
@ -446,7 +449,7 @@ LANGUAGE js AS
write={
"bigquery": "SELECT LAST_DAY(CAST('2008-11-25' AS DATE), MONTH)",
"duckdb": "SELECT LAST_DAY(CAST('2008-11-25' AS DATE))",
"clickhouse": "SELECT LAST_DAY(CAST('2008-11-25' AS DATE))",
"clickhouse": "SELECT LAST_DAY(CAST('2008-11-25' AS Nullable(DATE)))",
"mysql": "SELECT LAST_DAY(CAST('2008-11-25' AS DATE))",
"oracle": "SELECT LAST_DAY(CAST('2008-11-25' AS DATE))",
"postgres": "SELECT CAST(DATE_TRUNC('MONTH', CAST('2008-11-25' AS DATE)) + INTERVAL '1 MONTH' - INTERVAL '1 DAY' AS DATE)",
@ -510,6 +513,20 @@ LANGUAGE js AS
"duckdb": "SELECT STRFTIME(CAST('2023-12-25' AS DATE), '%Y%m%d')",
},
)
self.validate_all(
"SELECT FORMAT_DATETIME('%Y%m%d %H:%M:%S', DATETIME '2023-12-25 15:30:00')",
write={
"bigquery": "SELECT FORMAT_DATETIME('%Y%m%d %H:%M:%S', CAST('2023-12-25 15:30:00' AS DATETIME))",
"duckdb": "SELECT STRFTIME(CAST('2023-12-25 15:30:00' AS TIMESTAMP), '%Y%m%d %H:%M:%S')",
},
)
self.validate_all(
"SELECT FORMAT_DATETIME('%x', '2023-12-25 15:30:00')",
write={
"bigquery": "SELECT FORMAT_DATETIME('%x', '2023-12-25 15:30:00')",
"duckdb": "SELECT STRFTIME(CAST('2023-12-25 15:30:00' AS TIMESTAMP), '%x')",
},
)
self.validate_all(
"SELECT COUNTIF(x)",
read={
@ -636,6 +653,7 @@ LANGUAGE js AS
write={
"bigquery": "SELECT DATETIME_TRUNC('2023-01-01T01:01:01', HOUR)",
"databricks": "SELECT DATE_TRUNC('HOUR', '2023-01-01T01:01:01')",
"duckdb": "SELECT DATE_TRUNC('HOUR', CAST('2023-01-01T01:01:01' AS DATETIME))",
},
),
)
@ -1209,10 +1227,9 @@ LANGUAGE js AS
"SELECT * FROM a WHERE b IN UNNEST([1, 2, 3])",
write={
"bigquery": "SELECT * FROM a WHERE b IN UNNEST([1, 2, 3])",
"mysql": "SELECT * FROM a WHERE b IN (SELECT UNNEST(ARRAY(1, 2, 3)))",
"presto": "SELECT * FROM a WHERE b IN (SELECT UNNEST(ARRAY[1, 2, 3]))",
"hive": "SELECT * FROM a WHERE b IN (SELECT UNNEST(ARRAY(1, 2, 3)))",
"spark": "SELECT * FROM a WHERE b IN (SELECT UNNEST(ARRAY(1, 2, 3)))",
"hive": "SELECT * FROM a WHERE b IN (SELECT EXPLODE(ARRAY(1, 2, 3)))",
"spark": "SELECT * FROM a WHERE b IN (SELECT EXPLODE(ARRAY(1, 2, 3)))",
},
)
self.validate_all(
@ -1256,6 +1273,13 @@ LANGUAGE js AS
"starrocks": "DATE_DIFF('MINUTE', CAST('2010-07-07' AS DATE), CAST('2008-12-25' AS DATE))",
},
)
self.validate_all(
"DATE_DIFF('2021-01-01', '2020-01-01', DAY)",
write={
"bigquery": "DATE_DIFF('2021-01-01', '2020-01-01', DAY)",
"duckdb": "DATE_DIFF('DAY', CAST('2020-01-01' AS DATE), CAST('2021-01-01' AS DATE))",
},
)
self.validate_all(
"CURRENT_DATE('UTC')",
write={
@ -1402,6 +1426,57 @@ WHERE
"": "SELECT LENGTH(foo)",
},
)
self.validate_all(
"SELECT TIME_DIFF('12:00:00', '12:30:00', MINUTE)",
write={
"duckdb": "SELECT DATE_DIFF('MINUTE', CAST('12:30:00' AS TIME), CAST('12:00:00' AS TIME))",
"bigquery": "SELECT TIME_DIFF('12:00:00', '12:30:00', MINUTE)",
},
)
self.validate_all(
"ARRAY_CONCAT([1, 2], [3, 4], [5, 6])",
write={
"bigquery": "ARRAY_CONCAT([1, 2], [3, 4], [5, 6])",
"duckdb": "ARRAY_CONCAT([1, 2], ARRAY_CONCAT([3, 4], [5, 6]))",
"postgres": "ARRAY_CAT(ARRAY[1, 2], ARRAY_CAT(ARRAY[3, 4], ARRAY[5, 6]))",
"redshift": "ARRAY_CONCAT(ARRAY(1, 2), ARRAY_CONCAT(ARRAY(3, 4), ARRAY(5, 6)))",
"snowflake": "ARRAY_CAT([1, 2], ARRAY_CAT([3, 4], [5, 6]))",
"hive": "CONCAT(ARRAY(1, 2), ARRAY(3, 4), ARRAY(5, 6))",
"spark2": "CONCAT(ARRAY(1, 2), ARRAY(3, 4), ARRAY(5, 6))",
"spark": "CONCAT(ARRAY(1, 2), ARRAY(3, 4), ARRAY(5, 6))",
"databricks": "CONCAT(ARRAY(1, 2), ARRAY(3, 4), ARRAY(5, 6))",
"presto": "CONCAT(ARRAY[1, 2], ARRAY[3, 4], ARRAY[5, 6])",
"trino": "CONCAT(ARRAY[1, 2], ARRAY[3, 4], ARRAY[5, 6])",
},
)
self.validate_all(
"SELECT GENERATE_DATE_ARRAY('2016-10-05', '2016-10-08')",
write={
"duckdb": "SELECT CAST(GENERATE_SERIES(CAST('2016-10-05' AS DATE), CAST('2016-10-08' AS DATE), INTERVAL 1 DAY) AS DATE[])",
"bigquery": "SELECT GENERATE_DATE_ARRAY('2016-10-05', '2016-10-08', INTERVAL 1 DAY)",
},
)
self.validate_all(
"SELECT GENERATE_DATE_ARRAY('2016-10-05', '2016-10-08', INTERVAL '1' MONTH)",
write={
"duckdb": "SELECT CAST(GENERATE_SERIES(CAST('2016-10-05' AS DATE), CAST('2016-10-08' AS DATE), INTERVAL '1' MONTH) AS DATE[])",
"bigquery": "SELECT GENERATE_DATE_ARRAY('2016-10-05', '2016-10-08', INTERVAL '1' MONTH)",
},
)
self.validate_all(
"SELECT GENERATE_TIMESTAMP_ARRAY('2016-10-05 00:00:00', '2016-10-07 00:00:00', INTERVAL '1' DAY)",
write={
"duckdb": "SELECT GENERATE_SERIES(CAST('2016-10-05 00:00:00' AS TIMESTAMP), CAST('2016-10-07 00:00:00' AS TIMESTAMP), INTERVAL '1' DAY)",
"bigquery": "SELECT GENERATE_TIMESTAMP_ARRAY('2016-10-05 00:00:00', '2016-10-07 00:00:00', INTERVAL '1' DAY)",
},
)
self.validate_all(
"SELECT PARSE_DATE('%A %b %e %Y', 'Thursday Dec 25 2008')",
write={
"bigquery": "SELECT PARSE_DATE('%A %b %e %Y', 'Thursday Dec 25 2008')",
"duckdb": "SELECT CAST(STRPTIME('Thursday Dec 25 2008', '%A %b %-d %Y') AS DATE)",
},
)
def test_errors(self):
with self.assertRaises(TokenError):
@ -1794,14 +1869,14 @@ OPTIONS (
"SELECT * FROM UNNEST(ARRAY<STRUCT<x INT64>>[])",
write={
"bigquery": "SELECT * FROM UNNEST(CAST([] AS ARRAY<STRUCT<x INT64>>))",
"duckdb": "SELECT * FROM UNNEST(CAST([] AS STRUCT(x BIGINT)[]))",
"duckdb": "SELECT * FROM (SELECT UNNEST(CAST([] AS STRUCT(x BIGINT)[]), max_depth => 2))",
},
)
self.validate_all(
"SELECT * FROM UNNEST(ARRAY<STRUCT<device_id INT64, time DATETIME, signal INT64, state STRING>>[STRUCT(1, DATETIME '2023-11-01 09:34:01', 74, 'INACTIVE'),STRUCT(4, DATETIME '2023-11-01 09:38:01', 80, 'ACTIVE')])",
write={
"bigquery": "SELECT * FROM UNNEST(CAST([STRUCT(1, CAST('2023-11-01 09:34:01' AS DATETIME), 74, 'INACTIVE'), STRUCT(4, CAST('2023-11-01 09:38:01' AS DATETIME), 80, 'ACTIVE')] AS ARRAY<STRUCT<device_id INT64, time DATETIME, signal INT64, state STRING>>))",
"duckdb": "SELECT * FROM UNNEST(CAST([ROW(1, CAST('2023-11-01 09:34:01' AS TIMESTAMP), 74, 'INACTIVE'), ROW(4, CAST('2023-11-01 09:38:01' AS TIMESTAMP), 80, 'ACTIVE')] AS STRUCT(device_id BIGINT, time TIMESTAMP, signal BIGINT, state TEXT)[]))",
"duckdb": "SELECT * FROM (SELECT UNNEST(CAST([ROW(1, CAST('2023-11-01 09:34:01' AS TIMESTAMP), 74, 'INACTIVE'), ROW(4, CAST('2023-11-01 09:38:01' AS TIMESTAMP), 80, 'ACTIVE')] AS STRUCT(device_id BIGINT, time TIMESTAMP, signal BIGINT, state TEXT)[]), max_depth => 2))",
},
)
self.validate_all(
@ -1811,3 +1886,51 @@ OPTIONS (
"duckdb": "SELECT CAST(ROW(1, ROW('c_str')) AS STRUCT(a BIGINT, b STRUCT(c TEXT)))",
},
)
def test_convert(self):
for value, expected in [
(datetime.datetime(2023, 1, 1), "CAST('2023-01-01 00:00:00' AS DATETIME)"),
(datetime.datetime(2023, 1, 1, 12, 13, 14), "CAST('2023-01-01 12:13:14' AS DATETIME)"),
(
datetime.datetime(2023, 1, 1, 12, 13, 14, tzinfo=datetime.timezone.utc),
"CAST('2023-01-01 12:13:14+00:00' AS TIMESTAMP)",
),
(
pytz.timezone("America/Los_Angeles").localize(
datetime.datetime(2023, 1, 1, 12, 13, 14)
),
"CAST('2023-01-01 12:13:14-08:00' AS TIMESTAMP)",
),
]:
with self.subTest(value):
self.assertEqual(exp.convert(value).sql(dialect=self.dialect), expected)
def test_unnest(self):
self.validate_all(
"SELECT name, laps FROM UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps), STRUCT('Makhloufi' AS name, [24.5, 25.4, 26.6, 26.1] AS laps)])",
write={
"bigquery": "SELECT name, laps FROM UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps), STRUCT('Makhloufi' AS name, [24.5, 25.4, 26.6, 26.1] AS laps)])",
"duckdb": "SELECT name, laps FROM (SELECT UNNEST([{'name': 'Rudisha', 'laps': [23.4, 26.3, 26.4, 26.1]}, {'name': 'Makhloufi', 'laps': [24.5, 25.4, 26.6, 26.1]}], max_depth => 2))",
},
)
self.validate_all(
"WITH Races AS (SELECT '800M' AS race) SELECT race, name, laps FROM Races AS r CROSS JOIN UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps)])",
write={
"bigquery": "WITH Races AS (SELECT '800M' AS race) SELECT race, name, laps FROM Races AS r CROSS JOIN UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps)])",
"duckdb": "WITH Races AS (SELECT '800M' AS race) SELECT race, name, laps FROM Races AS r CROSS JOIN (SELECT UNNEST([{'name': 'Rudisha', 'laps': [23.4, 26.3, 26.4, 26.1]}], max_depth => 2))",
},
)
self.validate_all(
"SELECT participant FROM UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps)]) AS participant",
write={
"bigquery": "SELECT participant FROM UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps)]) AS participant",
"duckdb": "SELECT participant FROM (SELECT UNNEST([{'name': 'Rudisha', 'laps': [23.4, 26.3, 26.4, 26.1]}], max_depth => 2)) AS participant",
},
)
self.validate_all(
"WITH Races AS (SELECT '800M' AS race) SELECT race, participant FROM Races AS r CROSS JOIN UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps)]) AS participant",
write={
"bigquery": "WITH Races AS (SELECT '800M' AS race) SELECT race, participant FROM Races AS r CROSS JOIN UNNEST([STRUCT('Rudisha' AS name, [23.4, 26.3, 26.4, 26.1] AS laps)]) AS participant",
"duckdb": "WITH Races AS (SELECT '800M' AS race) SELECT race, participant FROM Races AS r CROSS JOIN (SELECT UNNEST([{'name': 'Rudisha', 'laps': [23.4, 26.3, 26.4, 26.1]}], max_depth => 2)) AS participant",
},
)

View file

@ -1,4 +1,7 @@
from datetime import date
from sqlglot import exp, parse_one
from sqlglot.dialects import ClickHouse
from sqlglot.expressions import convert
from tests.dialects.test_dialect import Validator
from sqlglot.errors import ErrorLevel
@ -7,27 +10,27 @@ class TestClickhouse(Validator):
dialect = "clickhouse"
def test_clickhouse(self):
self.validate_identity("SELECT toFloat(like)")
self.validate_identity("SELECT like")
for string_type_enum in ClickHouse.Generator.STRING_TYPE_MAPPING:
self.validate_identity(f"CAST(x AS {string_type_enum.value})", "CAST(x AS String)")
string_types = [
"BLOB",
"LONGBLOB",
"LONGTEXT",
"MEDIUMBLOB",
"MEDIUMTEXT",
"TINYBLOB",
"TINYTEXT",
"VARCHAR(255)",
]
# Arrays, maps and tuples can't be Nullable in ClickHouse
for non_nullable_type in ("ARRAY<INT>", "MAP<INT, INT>", "STRUCT(a: INT)"):
try_cast = parse_one(f"TRY_CAST(x AS {non_nullable_type})")
target_type = try_cast.to.sql("clickhouse")
self.assertEqual(try_cast.sql("clickhouse"), f"CAST(x AS {target_type})")
for string_type in string_types:
self.validate_identity(f"CAST(x AS {string_type})", "CAST(x AS String)")
for nullable_type in ("INT", "UINT", "BIGINT", "FLOAT", "DOUBLE", "TEXT", "DATE", "UUID"):
try_cast = parse_one(f"TRY_CAST(x AS {nullable_type})")
target_type = exp.DataType.build(nullable_type, dialect="clickhouse").sql("clickhouse")
self.assertEqual(try_cast.sql("clickhouse"), f"CAST(x AS Nullable({target_type}))")
expr = parse_one("count(x)")
self.assertEqual(expr.sql(dialect="clickhouse"), "COUNT(x)")
self.assertIsNone(expr._meta)
self.validate_identity("@macro").assert_is(exp.Parameter).this.assert_is(exp.Var)
self.validate_identity("SELECT toFloat(like)")
self.validate_identity("SELECT like")
self.validate_identity("SELECT STR_TO_DATE(str, fmt, tz)")
self.validate_identity("SELECT STR_TO_DATE('05 12 2000', '%d %m %Y')")
self.validate_identity("SELECT EXTRACT(YEAR FROM toDateTime('2023-02-01'))")
@ -40,7 +43,7 @@ class TestClickhouse(Validator):
self.validate_identity("SELECT * FROM (SELECT a FROM b SAMPLE 0.01)")
self.validate_identity("SELECT * FROM (SELECT a FROM b SAMPLE 1 / 10 OFFSET 1 / 2)")
self.validate_identity("SELECT sum(foo * bar) FROM bla SAMPLE 10000000")
self.validate_identity("CAST(x AS Nested(ID UInt32, Serial UInt32, EventTime DATETIME))")
self.validate_identity("CAST(x AS Nested(ID UInt32, Serial UInt32, EventTime DateTime))")
self.validate_identity("CAST(x AS Enum('hello' = 1, 'world' = 2))")
self.validate_identity("CAST(x AS Enum('hello', 'world'))")
self.validate_identity("CAST(x AS Enum('hello' = 1, 'world'))")
@ -79,7 +82,8 @@ class TestClickhouse(Validator):
self.validate_identity("SELECT * FROM foo WHERE x GLOBAL IN (SELECT * FROM bar)")
self.validate_identity("position(haystack, needle)")
self.validate_identity("position(haystack, needle, position)")
self.validate_identity("CAST(x AS DATETIME)")
self.validate_identity("CAST(x AS DATETIME)", "CAST(x AS DateTime)")
self.validate_identity("CAST(x AS TIMESTAMPTZ)", "CAST(x AS DateTime)")
self.validate_identity("CAST(x as MEDIUMINT)", "CAST(x AS Int32)")
self.validate_identity("SELECT arrayJoin([1, 2, 3] AS src) AS dst, 'Hello', src")
self.validate_identity("""SELECT JSONExtractString('{"x": {"y": 1}}', 'x', 'y')""")
@ -207,11 +211,16 @@ class TestClickhouse(Validator):
},
)
self.validate_all(
"SELECT CAST('2020-01-01' AS TIMESTAMP) + INTERVAL '500' MICROSECOND",
"SELECT CAST('2020-01-01' AS Nullable(DateTime)) + INTERVAL '500' MICROSECOND",
read={
"duckdb": "SELECT TIMESTAMP '2020-01-01' + INTERVAL '500 us'",
"postgres": "SELECT TIMESTAMP '2020-01-01' + INTERVAL '500 us'",
},
write={
"clickhouse": "SELECT CAST('2020-01-01' AS Nullable(DateTime)) + INTERVAL '500' MICROSECOND",
"duckdb": "SELECT CAST('2020-01-01' AS DATETIME) + INTERVAL '500' MICROSECOND",
"postgres": "SELECT CAST('2020-01-01' AS TIMESTAMP) + INTERVAL '500 MICROSECOND'",
},
)
self.validate_all(
"SELECT CURRENT_DATE()",
@ -471,6 +480,47 @@ class TestClickhouse(Validator):
parse_one("Tuple(select Int64)", into=exp.DataType, read="clickhouse"), exp.DataType
)
self.validate_identity("INSERT INTO t (col1, col2) VALUES ('abcd', 1234)")
self.validate_all(
"INSERT INTO t (col1, col2) VALUES ('abcd', 1234)",
read={
# looks like values table function, but should be parsed as VALUES block
"clickhouse": "INSERT INTO t (col1, col2) values('abcd', 1234)"
},
write={
"clickhouse": "INSERT INTO t (col1, col2) VALUES ('abcd', 1234)",
"postgres": "INSERT INTO t (col1, col2) VALUES ('abcd', 1234)",
},
)
def test_clickhouse_values(self):
values = exp.select("*").from_(
exp.values([exp.tuple_(1, 2, 3)], alias="subq", columns=["a", "b", "c"])
)
self.assertEqual(
values.sql("clickhouse"),
"SELECT * FROM (SELECT 1 AS a, 2 AS b, 3 AS c) AS subq",
)
self.validate_identity("INSERT INTO t (col1, col2) VALUES ('abcd', 1234)")
self.validate_identity(
"INSERT INTO t (col1, col2) FORMAT Values('abcd', 1234)",
"INSERT INTO t (col1, col2) VALUES ('abcd', 1234)",
)
self.validate_all(
"SELECT col FROM (SELECT 1 AS col) AS _t",
read={
"duckdb": "SELECT col FROM (VALUES (1)) AS _t(col)",
},
)
self.validate_all(
"SELECT col1, col2 FROM (SELECT 1 AS col1, 2 AS col2 UNION ALL SELECT 3, 4) AS _t",
read={
"duckdb": "SELECT col1, col2 FROM (VALUES (1, 2), (3, 4)) AS _t(col1, col2)",
},
)
def test_cte(self):
self.validate_identity("WITH 'x' AS foo SELECT foo")
self.validate_identity("WITH ['c'] AS field_names SELECT field_names")
@ -531,7 +581,7 @@ class TestClickhouse(Validator):
self.validate_all(
"SELECT {abc: UInt32}, {b: String}, {c: DateTime},{d: Map(String, Array(UInt8))}, {e: Tuple(UInt8, String)}",
write={
"clickhouse": "SELECT {abc: UInt32}, {b: String}, {c: DATETIME}, {d: Map(String, Array(UInt8))}, {e: Tuple(UInt8, String)}",
"clickhouse": "SELECT {abc: UInt32}, {b: String}, {c: DateTime}, {d: Map(String, Array(UInt8))}, {e: Tuple(UInt8, String)}",
"": "SELECT :abc, :b, :c, :d, :e",
},
)
@ -562,14 +612,77 @@ class TestClickhouse(Validator):
)
def test_ddl(self):
db_table_expr = exp.Table(this=None, db=exp.to_identifier("foo"), catalog=None)
create_with_cluster = exp.Create(
this=db_table_expr,
kind="DATABASE",
properties=exp.Properties(expressions=[exp.OnCluster(this=exp.to_identifier("c"))]),
)
self.assertEqual(create_with_cluster.sql("clickhouse"), "CREATE DATABASE foo ON CLUSTER c")
ctas_with_comment = exp.Create(
this=exp.table_("foo"),
kind="TABLE",
expression=exp.select("*").from_("db.other_table"),
properties=exp.Properties(
expressions=[
exp.EngineProperty(this=exp.var("Memory")),
exp.SchemaCommentProperty(this=exp.Literal.string("foo")),
],
),
)
self.assertEqual(
ctas_with_comment.sql("clickhouse"),
"CREATE TABLE foo ENGINE=Memory AS (SELECT * FROM db.other_table) COMMENT 'foo'",
)
self.validate_identity("""CREATE TABLE ip_data (ip4 IPv4, ip6 IPv6) ENGINE=TinyLog()""")
self.validate_identity("""CREATE TABLE dates (dt1 Date32) ENGINE=TinyLog()""")
self.validate_identity("CREATE TABLE named_tuples (a Tuple(select String, i Int64))")
self.validate_identity("""CREATE TABLE t (a String) EMPTY AS SELECT * FROM dummy""")
self.validate_identity(
"CREATE TABLE t1 (a String EPHEMERAL, b String EPHEMERAL func(), c String MATERIALIZED func(), d String ALIAS func()) ENGINE=TinyLog()"
)
self.validate_identity(
"CREATE TABLE t (a String, b String, c UInt64, PROJECTION p1 (SELECT a, sum(c) GROUP BY a, b), PROJECTION p2 (SELECT b, sum(c) GROUP BY b)) ENGINE=MergeTree()"
)
self.validate_identity(
"""CREATE TABLE xyz (ts DateTime, data String) ENGINE=MergeTree() ORDER BY ts SETTINGS index_granularity = 8192 COMMENT '{"key": "value"}'"""
)
self.validate_identity(
"INSERT INTO FUNCTION s3('a', 'b', 'c', 'd', 'e') PARTITION BY CONCAT(s1, s2, s3, s4) SETTINGS set1 = 1, set2 = '2' SELECT * FROM some_table SETTINGS foo = 3"
)
self.validate_identity(
'CREATE TABLE data5 ("x" UInt32, "y" UInt32) ENGINE=MergeTree ORDER BY (round(y / 1000000000), cityHash64(x)) SAMPLE BY cityHash64(x)'
)
self.validate_identity(
"CREATE TABLE foo (x UInt32) TTL time_column + INTERVAL '1' MONTH DELETE WHERE column = 'value'"
)
self.validate_identity("CREATE TABLE named_tuples (a Tuple(select String, i Int64))")
self.validate_identity(
"CREATE TABLE a ENGINE=Memory AS SELECT 1 AS c COMMENT 'foo'",
"CREATE TABLE a ENGINE=Memory AS (SELECT 1 AS c) COMMENT 'foo'",
)
self.validate_all(
"CREATE DATABASE x",
read={
"duckdb": "CREATE SCHEMA x",
},
write={
"clickhouse": "CREATE DATABASE x",
"duckdb": "CREATE SCHEMA x",
},
)
self.validate_all(
"DROP DATABASE x",
read={
"duckdb": "DROP SCHEMA x",
},
write={
"clickhouse": "DROP DATABASE x",
"duckdb": "DROP SCHEMA x",
},
)
self.validate_all(
"""
CREATE TABLE example1 (
@ -583,7 +696,7 @@ class TestClickhouse(Validator):
""",
write={
"clickhouse": """CREATE TABLE example1 (
timestamp DATETIME,
timestamp DateTime,
x UInt32 TTL now() + INTERVAL '1' MONTH,
y String TTL timestamp + INTERVAL '1' DAY,
z String
@ -661,7 +774,7 @@ SETTINGS
""",
write={
"clickhouse": """CREATE TABLE example_table (
d DATETIME,
d DateTime,
a Int32
)
ENGINE=MergeTree
@ -688,7 +801,7 @@ TTL
""",
write={
"clickhouse": """CREATE TABLE table_with_where (
d DATETIME,
d DateTime,
a Int32
)
ENGINE=MergeTree
@ -716,7 +829,7 @@ WHERE
""",
write={
"clickhouse": """CREATE TABLE table_for_recompression (
d DATETIME,
d DateTime,
key UInt64,
value String
)
@ -748,7 +861,7 @@ SETTINGS
""",
write={
"clickhouse": """CREATE TABLE table_for_aggregation (
d DATETIME,
d DateTime,
k1 Int32,
k2 Int32,
x Int32,
@ -855,8 +968,6 @@ LIFETIME(MIN 0 MAX 0)""",
},
pretty=True,
)
self.validate_identity("""CREATE TABLE ip_data (ip4 IPv4, ip6 IPv6) ENGINE=TinyLog()""")
self.validate_identity("""CREATE TABLE dates (dt1 Date32) ENGINE=TinyLog()""")
self.validate_all(
"""
CREATE TABLE t (
@ -873,12 +984,6 @@ LIFETIME(MIN 0 MAX 0)""",
},
pretty=True,
)
self.validate_identity(
"CREATE TABLE t1 (a String EPHEMERAL, b String EPHEMERAL func(), c String MATERIALIZED func(), d String ALIAS func()) ENGINE=TinyLog()"
)
self.validate_identity(
"CREATE TABLE t (a String, b String, c UInt64, PROJECTION p1 (SELECT a, sum(c) GROUP BY a, b), PROJECTION p2 (SELECT b, sum(c) GROUP BY b)) ENGINE=MergeTree()"
)
def test_agg_functions(self):
def extract_agg_func(query):
@ -934,3 +1039,8 @@ LIFETIME(MIN 0 MAX 0)""",
f"SELECT {func_alias}(SECOND, 1, bar)",
f"SELECT {func_name}(SECOND, 1, bar)",
)
def test_convert(self):
self.assertEqual(
convert(date(2020, 1, 1)).sql(dialect=self.dialect), "toDate('2020-01-01')"
)

View file

@ -256,3 +256,11 @@ class TestDatabricks(Validator):
"databricks": "WITH x AS (SELECT 1) SELECT * FROM x",
},
)
def test_streaming_tables(self):
self.validate_identity(
"CREATE STREAMING TABLE raw_data AS SELECT * FROM STREAM READ_FILES('abfss://container@storageAccount.dfs.core.windows.net/base/path')"
)
self.validate_identity(
"CREATE OR REFRESH STREAMING TABLE csv_data (id INT, ts TIMESTAMP, event STRING) AS SELECT * FROM STREAM READ_FILES('s3://bucket/path', format => 'csv', schema => 'id int, ts timestamp, event string')"
)

View file

@ -160,7 +160,7 @@ class TestDialect(Validator):
"CAST(a AS TEXT)",
write={
"bigquery": "CAST(a AS STRING)",
"clickhouse": "CAST(a AS String)",
"clickhouse": "CAST(a AS Nullable(String))",
"drill": "CAST(a AS VARCHAR)",
"duckdb": "CAST(a AS TEXT)",
"materialize": "CAST(a AS TEXT)",
@ -181,7 +181,7 @@ class TestDialect(Validator):
"CAST(a AS BINARY(4))",
write={
"bigquery": "CAST(a AS BYTES)",
"clickhouse": "CAST(a AS BINARY(4))",
"clickhouse": "CAST(a AS Nullable(BINARY(4)))",
"drill": "CAST(a AS VARBINARY(4))",
"duckdb": "CAST(a AS BLOB(4))",
"materialize": "CAST(a AS BYTEA(4))",
@ -201,7 +201,7 @@ class TestDialect(Validator):
"CAST(a AS VARBINARY(4))",
write={
"bigquery": "CAST(a AS BYTES)",
"clickhouse": "CAST(a AS String)",
"clickhouse": "CAST(a AS Nullable(String))",
"duckdb": "CAST(a AS BLOB(4))",
"materialize": "CAST(a AS BYTEA(4))",
"mysql": "CAST(a AS VARBINARY(4))",
@ -219,19 +219,19 @@ class TestDialect(Validator):
self.validate_all(
"CAST(MAP('a', '1') AS MAP(TEXT, TEXT))",
write={
"clickhouse": "CAST(map('a', '1') AS Map(String, String))",
"clickhouse": "CAST(map('a', '1') AS Map(String, Nullable(String)))",
},
)
self.validate_all(
"CAST(ARRAY(1, 2) AS ARRAY<TINYINT>)",
write={
"clickhouse": "CAST([1, 2] AS Array(Int8))",
"clickhouse": "CAST([1, 2] AS Array(Nullable(Int8)))",
},
)
self.validate_all(
"CAST((1, 2) AS STRUCT<a: TINYINT, b: SMALLINT, c: INT, d: BIGINT>)",
"CAST((1, 2, 3, 4) AS STRUCT<a: TINYINT, b: SMALLINT, c: INT, d: BIGINT>)",
write={
"clickhouse": "CAST((1, 2) AS Tuple(a Int8, b Int16, c Int32, d Int64))",
"clickhouse": "CAST((1, 2, 3, 4) AS Tuple(a Nullable(Int8), b Nullable(Int16), c Nullable(Int32), d Nullable(Int64)))",
},
)
self.validate_all(
@ -327,20 +327,10 @@ class TestDialect(Validator):
"postgres": "CAST(a AS DOUBLE PRECISION)",
"redshift": "CAST(a AS DOUBLE PRECISION)",
},
write={
"duckdb": "CAST(a AS DOUBLE)",
"drill": "CAST(a AS DOUBLE)",
"postgres": "CAST(a AS DOUBLE PRECISION)",
"redshift": "CAST(a AS DOUBLE PRECISION)",
"doris": "CAST(a AS DOUBLE)",
},
)
self.validate_all(
"CAST(a AS DOUBLE)",
write={
"bigquery": "CAST(a AS FLOAT64)",
"clickhouse": "CAST(a AS Float64)",
"clickhouse": "CAST(a AS Nullable(Float64))",
"doris": "CAST(a AS DOUBLE)",
"drill": "CAST(a AS DOUBLE)",
"duckdb": "CAST(a AS DOUBLE)",
"materialize": "CAST(a AS DOUBLE PRECISION)",
@ -592,7 +582,7 @@ class TestDialect(Validator):
"hive": "CAST(FROM_UNIXTIME(UNIX_TIMESTAMP(x, 'yyyy-MM-ddTHH:mm:ss')) AS TIMESTAMP)",
"presto": "DATE_PARSE(x, '%Y-%m-%dT%T')",
"drill": "TO_TIMESTAMP(x, 'yyyy-MM-dd''T''HH:mm:ss')",
"redshift": "TO_TIMESTAMP(x, 'YYYY-MM-DDTHH:MI:SS')",
"redshift": "TO_TIMESTAMP(x, 'YYYY-MM-DDTHH24:MI:SS')",
"spark": "TO_TIMESTAMP(x, 'yyyy-MM-ddTHH:mm:ss')",
},
)
@ -647,14 +637,66 @@ class TestDialect(Validator):
self.validate_all(
"TIME_STR_TO_TIME('2020-01-01')",
write={
"drill": "CAST('2020-01-01' AS TIMESTAMP)",
"bigquery": "CAST('2020-01-01' AS DATETIME)",
"databricks": "CAST('2020-01-01' AS TIMESTAMP)",
"duckdb": "CAST('2020-01-01' AS TIMESTAMP)",
"tsql": "CAST('2020-01-01' AS DATETIME2)",
"mysql": "CAST('2020-01-01' AS DATETIME)",
"postgres": "CAST('2020-01-01' AS TIMESTAMP)",
"redshift": "CAST('2020-01-01' AS TIMESTAMP)",
"snowflake": "CAST('2020-01-01' AS TIMESTAMP)",
"spark": "CAST('2020-01-01' AS TIMESTAMP)",
"trino": "CAST('2020-01-01' AS TIMESTAMP)",
"clickhouse": "CAST('2020-01-01' AS Nullable(DateTime))",
"drill": "CAST('2020-01-01' AS TIMESTAMP)",
"hive": "CAST('2020-01-01' AS TIMESTAMP)",
"presto": "CAST('2020-01-01' AS TIMESTAMP)",
"sqlite": "'2020-01-01'",
"doris": "CAST('2020-01-01' AS DATETIME)",
},
)
self.validate_all(
"TIME_STR_TO_TIME('2020-01-01 12:13:14-08:00', 'America/Los_Angeles')",
write={
"bigquery": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP)",
"databricks": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP)",
"duckdb": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMPTZ)",
"tsql": "CAST('2020-01-01 12:13:14-08:00' AS DATETIMEOFFSET) AT TIME ZONE 'UTC'",
"mysql": "CAST('2020-01-01 12:13:14-08:00' AS DATETIME)",
"postgres": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMPTZ)",
"redshift": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP WITH TIME ZONE)",
"snowflake": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMPTZ)",
"spark": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP)",
"trino": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP WITH TIME ZONE)",
"clickhouse": "CAST('2020-01-01 12:13:14' AS Nullable(DateTime('America/Los_Angeles')))",
"drill": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP)",
"hive": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP)",
"presto": "CAST('2020-01-01 12:13:14-08:00' AS TIMESTAMP WITH TIME ZONE)",
"sqlite": "'2020-01-01 12:13:14-08:00'",
"doris": "CAST('2020-01-01 12:13:14-08:00' AS DATETIME)",
},
)
self.validate_all(
"TIME_STR_TO_TIME(col, 'America/Los_Angeles')",
write={
"bigquery": "CAST(col AS TIMESTAMP)",
"databricks": "CAST(col AS TIMESTAMP)",
"duckdb": "CAST(col AS TIMESTAMPTZ)",
"tsql": "CAST(col AS DATETIMEOFFSET) AT TIME ZONE 'UTC'",
"mysql": "CAST(col AS DATETIME)",
"postgres": "CAST(col AS TIMESTAMPTZ)",
"redshift": "CAST(col AS TIMESTAMP WITH TIME ZONE)",
"snowflake": "CAST(col AS TIMESTAMPTZ)",
"spark": "CAST(col AS TIMESTAMP)",
"trino": "CAST(col AS TIMESTAMP WITH TIME ZONE)",
"clickhouse": "CAST(col AS Nullable(DateTime('America/Los_Angeles')))",
"drill": "CAST(col AS TIMESTAMP)",
"hive": "CAST(col AS TIMESTAMP)",
"presto": "CAST(col AS TIMESTAMP WITH TIME ZONE)",
"sqlite": "col",
"doris": "CAST(col AS DATETIME)",
},
)
self.validate_all(
"TIME_STR_TO_UNIX('2020-01-01')",
write={
@ -2115,6 +2157,17 @@ SELECT
},
)
# needs to preserve the target alias in then WHEN condition but not in the THEN clause
self.validate_all(
"""MERGE INTO foo AS target USING (SELECT a, b FROM tbl) AS src ON src.a = target.a
WHEN MATCHED AND target.a <> src.a THEN UPDATE SET target.b = 'FOO'
WHEN NOT MATCHED THEN INSERT (target.a, target.b) VALUES (src.a, src.b)""",
write={
"trino": """MERGE INTO foo AS target USING (SELECT a, b FROM tbl) AS src ON src.a = target.a WHEN MATCHED AND target.a <> src.a THEN UPDATE SET b = 'FOO' WHEN NOT MATCHED THEN INSERT (a, b) VALUES (src.a, src.b)""",
"postgres": """MERGE INTO foo AS target USING (SELECT a, b FROM tbl) AS src ON src.a = target.a WHEN MATCHED AND target.a <> src.a THEN UPDATE SET b = 'FOO' WHEN NOT MATCHED THEN INSERT (a, b) VALUES (src.a, src.b)""",
},
)
def test_substring(self):
self.validate_all(
"SUBSTR('123456', 2, 3)",
@ -2603,3 +2656,46 @@ FROM subquery2""",
"trino": f"SELECT {pad_func}('bar', 5, ' ')",
},
)
def test_generate_date_array(self):
self.validate_all(
"SELECT * FROM UNNEST(GENERATE_DATE_ARRAY(DATE '2020-01-01', DATE '2020-02-01', INTERVAL 1 WEEK))",
write={
"bigquery": "SELECT * FROM UNNEST(GENERATE_DATE_ARRAY(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), INTERVAL '1' WEEK))",
"databricks": "SELECT * FROM EXPLODE(SEQUENCE(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), INTERVAL '1' WEEK))",
"duckdb": "SELECT * FROM UNNEST(CAST(GENERATE_SERIES(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), (7 * INTERVAL '1' DAY)) AS DATE[]))",
"mysql": "WITH RECURSIVE _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATE_ADD(date_value, INTERVAL 1 WEEK) AS DATE) FROM _generated_dates WHERE CAST(DATE_ADD(date_value, INTERVAL 1 WEEK) AS DATE) <= CAST('2020-02-01' AS DATE)) SELECT * FROM (SELECT date_value FROM _generated_dates) AS _generated_dates",
"postgres": "SELECT * FROM (SELECT CAST(value AS DATE) FROM GENERATE_SERIES(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), INTERVAL '1 WEEK') AS value) AS _unnested_generate_series",
"presto": "SELECT * FROM UNNEST(SEQUENCE(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), (1 * INTERVAL '7' DAY)))",
"redshift": "WITH RECURSIVE _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_value) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_value) AS DATE) <= CAST('2020-02-01' AS DATE)) SELECT * FROM (SELECT date_value FROM _generated_dates) AS _generated_dates",
"snowflake": "SELECT * FROM (SELECT DATEADD(WEEK, CAST(value AS INT), CAST('2020-01-01' AS DATE)) AS value FROM TABLE(FLATTEN(INPUT => ARRAY_GENERATE_RANGE(0, (DATEDIFF(WEEK, CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE)) + 1 - 1) + 1))) AS _u(seq, key, path, index, value, this))",
"spark": "SELECT * FROM EXPLODE(SEQUENCE(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), INTERVAL '1' WEEK))",
"trino": "SELECT * FROM UNNEST(SEQUENCE(CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE), (1 * INTERVAL '7' DAY)))",
"tsql": "WITH _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_value) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_value) AS DATE) <= CAST('2020-02-01' AS DATE)) SELECT * FROM (SELECT date_value AS date_value FROM _generated_dates) AS _generated_dates",
},
)
self.validate_all(
"WITH dates AS (SELECT * FROM UNNEST(GENERATE_DATE_ARRAY(DATE '2020-01-01', DATE '2020-02-01', INTERVAL 1 WEEK))) SELECT * FROM dates",
write={
"mysql": "WITH RECURSIVE _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATE_ADD(date_value, INTERVAL 1 WEEK) AS DATE) FROM _generated_dates WHERE CAST(DATE_ADD(date_value, INTERVAL 1 WEEK) AS DATE) <= CAST('2020-02-01' AS DATE)), dates AS (SELECT * FROM (SELECT date_value FROM _generated_dates) AS _generated_dates) SELECT * FROM dates",
"redshift": "WITH RECURSIVE _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_value) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_value) AS DATE) <= CAST('2020-02-01' AS DATE)), dates AS (SELECT * FROM (SELECT date_value FROM _generated_dates) AS _generated_dates) SELECT * FROM dates",
"tsql": "WITH _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_value) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_value) AS DATE) <= CAST('2020-02-01' AS DATE)), dates AS (SELECT * FROM (SELECT date_value AS date_value FROM _generated_dates) AS _generated_dates) SELECT * FROM dates",
},
)
self.validate_all(
"WITH dates1 AS (SELECT * FROM UNNEST(GENERATE_DATE_ARRAY(DATE '2020-01-01', DATE '2020-02-01', INTERVAL 1 WEEK))), dates2 AS (SELECT * FROM UNNEST(GENERATE_DATE_ARRAY(DATE '2020-01-01', DATE '2020-03-01', INTERVAL 1 MONTH))) SELECT * FROM dates1 CROSS JOIN dates2",
write={
"mysql": "WITH RECURSIVE _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATE_ADD(date_value, INTERVAL 1 WEEK) AS DATE) FROM _generated_dates WHERE CAST(DATE_ADD(date_value, INTERVAL 1 WEEK) AS DATE) <= CAST('2020-02-01' AS DATE)), _generated_dates_1(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATE_ADD(date_value, INTERVAL 1 MONTH) AS DATE) FROM _generated_dates_1 WHERE CAST(DATE_ADD(date_value, INTERVAL 1 MONTH) AS DATE) <= CAST('2020-03-01' AS DATE)), dates1 AS (SELECT * FROM (SELECT date_value FROM _generated_dates) AS _generated_dates), dates2 AS (SELECT * FROM (SELECT date_value FROM _generated_dates_1) AS _generated_dates_1) SELECT * FROM dates1 CROSS JOIN dates2",
"redshift": "WITH RECURSIVE _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_value) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_value) AS DATE) <= CAST('2020-02-01' AS DATE)), _generated_dates_1(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(MONTH, 1, date_value) AS DATE) FROM _generated_dates_1 WHERE CAST(DATEADD(MONTH, 1, date_value) AS DATE) <= CAST('2020-03-01' AS DATE)), dates1 AS (SELECT * FROM (SELECT date_value FROM _generated_dates) AS _generated_dates), dates2 AS (SELECT * FROM (SELECT date_value FROM _generated_dates_1) AS _generated_dates_1) SELECT * FROM dates1 CROSS JOIN dates2",
"tsql": "WITH _generated_dates(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_value) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_value) AS DATE) <= CAST('2020-02-01' AS DATE)), _generated_dates_1(date_value) AS (SELECT CAST('2020-01-01' AS DATE) AS date_value UNION ALL SELECT CAST(DATEADD(MONTH, 1, date_value) AS DATE) FROM _generated_dates_1 WHERE CAST(DATEADD(MONTH, 1, date_value) AS DATE) <= CAST('2020-03-01' AS DATE)), dates1 AS (SELECT * FROM (SELECT date_value AS date_value FROM _generated_dates) AS _generated_dates), dates2 AS (SELECT * FROM (SELECT date_value AS date_value FROM _generated_dates_1) AS _generated_dates_1) SELECT * FROM dates1 CROSS JOIN dates2",
},
)
self.validate_all(
"SELECT * FROM UNNEST(GENERATE_DATE_ARRAY(DATE '2020-01-01', DATE '2020-02-01', INTERVAL 1 WEEK)) AS _q(date_week)",
write={
"mysql": "WITH RECURSIVE _generated_dates(date_week) AS (SELECT CAST('2020-01-01' AS DATE) AS date_week UNION ALL SELECT CAST(DATE_ADD(date_week, INTERVAL 1 WEEK) AS DATE) FROM _generated_dates WHERE CAST(DATE_ADD(date_week, INTERVAL 1 WEEK) AS DATE) <= CAST('2020-02-01' AS DATE)) SELECT * FROM (SELECT date_week FROM _generated_dates) AS _generated_dates",
"redshift": "WITH RECURSIVE _generated_dates(date_week) AS (SELECT CAST('2020-01-01' AS DATE) AS date_week UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_week) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_week) AS DATE) <= CAST('2020-02-01' AS DATE)) SELECT * FROM (SELECT date_week FROM _generated_dates) AS _generated_dates",
"snowflake": "SELECT * FROM (SELECT DATEADD(WEEK, CAST(date_week AS INT), CAST('2020-01-01' AS DATE)) AS date_week FROM TABLE(FLATTEN(INPUT => ARRAY_GENERATE_RANGE(0, (DATEDIFF(WEEK, CAST('2020-01-01' AS DATE), CAST('2020-02-01' AS DATE)) + 1 - 1) + 1))) AS _q(seq, key, path, index, date_week, this)) AS _q(date_week)",
"tsql": "WITH _generated_dates(date_week) AS (SELECT CAST('2020-01-01' AS DATE) AS date_week UNION ALL SELECT CAST(DATEADD(WEEK, 1, date_week) AS DATE) FROM _generated_dates WHERE CAST(DATEADD(WEEK, 1, date_week) AS DATE) <= CAST('2020-02-01' AS DATE)) SELECT * FROM (SELECT date_week AS date_week FROM _generated_dates) AS _generated_dates",
},
)

View file

@ -8,8 +8,6 @@ class TestDuckDB(Validator):
dialect = "duckdb"
def test_duckdb(self):
self.validate_identity("x::int[3]", "CAST(x AS INT[3])")
with self.assertRaises(ParseError):
parse_one("1 //", read="duckdb")
@ -34,31 +32,6 @@ class TestDuckDB(Validator):
"mysql": "SELECT `straight_join`",
},
)
self.validate_all(
"SELECT CAST('2020-01-01 12:05:01' AS TIMESTAMP)",
read={
"duckdb": "SELECT CAST('2020-01-01 12:05:01' AS TIMESTAMP)",
"snowflake": "SELECT CAST('2020-01-01 12:05:01' AS TIMESTAMPNTZ)",
},
)
self.validate_all(
"SELECT CAST('2020-01-01' AS DATE) + INTERVAL (day_offset) DAY FROM t",
read={
"duckdb": "SELECT CAST('2020-01-01' AS DATE) + INTERVAL (day_offset) DAY FROM t",
"mysql": "SELECT DATE '2020-01-01' + INTERVAL day_offset DAY FROM t",
},
)
self.validate_all(
"SELECT CAST('09:05:03' AS TIME) + INTERVAL 2 HOUR",
read={
"bigquery": "SELECT TIME_ADD(CAST('09:05:03' AS TIME), INTERVAL 2 HOUR)",
"snowflake": "SELECT TIMEADD(HOUR, 2, TO_TIME('09:05:03'))",
},
write={
"duckdb": "SELECT CAST('09:05:03' AS TIME) + INTERVAL '2' HOUR",
"snowflake": "SELECT CAST('09:05:03' AS TIME) + INTERVAL '2 HOUR'",
},
)
self.validate_all(
'STRUCT_PACK("a b" := 1)',
write={
@ -112,7 +85,9 @@ class TestDuckDB(Validator):
self.validate_all(
"CREATE TEMPORARY FUNCTION f1(a, b) AS (a + b)",
read={"bigquery": "CREATE TEMP FUNCTION f1(a INT64, b INT64) AS (a + b)"},
read={
"bigquery": "CREATE TEMP FUNCTION f1(a INT64, b INT64) AS (a + b)",
},
)
self.validate_identity("SELECT 1 WHERE x > $1")
self.validate_identity("SELECT 1 WHERE x > $name")
@ -128,13 +103,17 @@ class TestDuckDB(Validator):
)
self.validate_all(
"{'a': 1, 'b': '2'}", write={"presto": "CAST(ROW(1, '2') AS ROW(a INTEGER, b VARCHAR))"}
"{'a': 1, 'b': '2'}",
write={
"presto": "CAST(ROW(1, '2') AS ROW(a INTEGER, b VARCHAR))",
},
)
self.validate_all(
"struct_pack(a := 1, b := 2)",
write={"presto": "CAST(ROW(1, 2) AS ROW(a INTEGER, b INTEGER))"},
write={
"presto": "CAST(ROW(1, 2) AS ROW(a INTEGER, b INTEGER))",
},
)
self.validate_all(
"struct_pack(a := 1, b := x)",
write={
@ -818,6 +797,9 @@ class TestDuckDB(Validator):
)
self.validate_identity("SELECT LENGTH(foo)")
self.validate_identity("SELECT ARRAY[1, 2, 3]", "SELECT [1, 2, 3]")
self.validate_identity("SELECT * FROM (DESCRIBE t)")
def test_array_index(self):
with self.assertLogs(helper_logger) as cm:
@ -909,12 +891,12 @@ class TestDuckDB(Validator):
"EPOCH_MS(x)",
write={
"bigquery": "TIMESTAMP_MILLIS(x)",
"clickhouse": "fromUnixTimestamp64Milli(CAST(x AS Nullable(Int64)))",
"duckdb": "EPOCH_MS(x)",
"mysql": "FROM_UNIXTIME(x / POWER(10, 3))",
"postgres": "TO_TIMESTAMP(CAST(x AS DOUBLE PRECISION) / 10 ^ 3)",
"presto": "FROM_UNIXTIME(CAST(x AS DOUBLE) / POW(10, 3))",
"spark": "TIMESTAMP_MILLIS(x)",
"clickhouse": "fromUnixTimestamp64Milli(CAST(x AS Int64))",
"postgres": "TO_TIMESTAMP(CAST(x AS DOUBLE PRECISION) / 10 ^ 3)",
"mysql": "FROM_UNIXTIME(x / POWER(10, 3))",
},
)
self.validate_all(
@ -958,7 +940,7 @@ class TestDuckDB(Validator):
self.validate_all(
"STRPTIME(x, '%-m/%-d/%y %-I:%M %p')",
write={
"bigquery": "PARSE_TIMESTAMP('%-m/%-d/%y %-I:%M %p', x)",
"bigquery": "PARSE_TIMESTAMP('%-m/%e/%y %-I:%M %p', x)",
"duckdb": "STRPTIME(x, '%-m/%-d/%y %-I:%M %p')",
"presto": "DATE_PARSE(x, '%c/%e/%y %l:%i %p')",
"hive": "CAST(FROM_UNIXTIME(UNIX_TIMESTAMP(x, 'M/d/yy h:mm a')) AS TIMESTAMP)",
@ -1023,6 +1005,7 @@ class TestDuckDB(Validator):
self.validate_identity("ARRAY((SELECT id FROM t))")
def test_cast(self):
self.validate_identity("x::int[3]", "CAST(x AS INT[3])")
self.validate_identity("CAST(x AS REAL)")
self.validate_identity("CAST(x AS UINTEGER)")
self.validate_identity("CAST(x AS UBIGINT)")
@ -1076,6 +1059,39 @@ class TestDuckDB(Validator):
"STRUCT_PACK(a := 'b')::STRUCT(a TEXT)",
"CAST(ROW('b') AS STRUCT(a TEXT))",
)
self.validate_all(
"CAST(x AS TIME)",
read={
"duckdb": "CAST(x AS TIME)",
"presto": "CAST(x AS TIME(6))",
},
)
self.validate_all(
"SELECT CAST('2020-01-01 12:05:01' AS TIMESTAMP)",
read={
"duckdb": "SELECT CAST('2020-01-01 12:05:01' AS TIMESTAMP)",
"snowflake": "SELECT CAST('2020-01-01 12:05:01' AS TIMESTAMPNTZ)",
},
)
self.validate_all(
"SELECT CAST('2020-01-01' AS DATE) + INTERVAL (day_offset) DAY FROM t",
read={
"duckdb": "SELECT CAST('2020-01-01' AS DATE) + INTERVAL (day_offset) DAY FROM t",
"mysql": "SELECT DATE '2020-01-01' + INTERVAL day_offset DAY FROM t",
},
)
self.validate_all(
"SELECT CAST('09:05:03' AS TIME) + INTERVAL 2 HOUR",
read={
"bigquery": "SELECT TIME_ADD(CAST('09:05:03' AS TIME), INTERVAL 2 HOUR)",
"snowflake": "SELECT TIMEADD(HOUR, 2, TO_TIME('09:05:03'))",
},
write={
"duckdb": "SELECT CAST('09:05:03' AS TIME) + INTERVAL '2' HOUR",
"snowflake": "SELECT CAST('09:05:03' AS TIME) + INTERVAL '2 HOUR'",
},
)
self.validate_all(
"CAST(x AS VARCHAR(5))",
write={
@ -1156,6 +1172,12 @@ class TestDuckDB(Validator):
},
)
self.validate_identity("SELECT x::INT[3][3]", "SELECT CAST(x AS INT[3][3])")
self.validate_identity(
"""SELECT ARRAY[[[1]]]::INT[1][1][1]""",
"""SELECT CAST([[[1]]] AS INT[1][1][1])""",
)
def test_encode_decode(self):
self.validate_all(
"ENCODE(x)",

View file

@ -171,6 +171,16 @@ class TestHive(Validator):
self.validate_identity(
"""CREATE EXTERNAL TABLE `my_table` (`a7` ARRAY<DATE>) ROW FORMAT SERDE 'a' STORED AS INPUTFORMAT 'b' OUTPUTFORMAT 'c' LOCATION 'd' TBLPROPERTIES ('e'='f')"""
)
self.validate_identity("ALTER VIEW v1 AS SELECT x, UPPER(s) AS s FROM t2")
self.validate_identity("ALTER VIEW v1 (c1, c2) AS SELECT x, UPPER(s) AS s FROM t2")
self.validate_identity(
"ALTER VIEW v7 (c1 COMMENT 'Comment for c1', c2) AS SELECT t1.c1, t1.c2 FROM t1"
)
self.validate_identity("ALTER VIEW db1.v1 RENAME TO db2.v2")
self.validate_identity("ALTER VIEW v1 SET TBLPROPERTIES ('tblp1'='1', 'tblp2'='2')")
self.validate_identity(
"ALTER VIEW v1 UNSET TBLPROPERTIES ('tblp1', 'tblp2')", check_command_warning=True
)
def test_lateral_view(self):
self.validate_all(

View file

@ -24,6 +24,18 @@ class TestMySQL(Validator):
self.validate_identity("ALTER TABLE t ADD INDEX `i` (`c`)")
self.validate_identity("ALTER TABLE t ADD UNIQUE `i` (`c`)")
self.validate_identity("ALTER TABLE test_table MODIFY COLUMN test_column LONGTEXT")
self.validate_identity("ALTER VIEW v AS SELECT a, b, c, d FROM foo")
self.validate_identity("ALTER VIEW v AS SELECT * FROM foo WHERE c > 100")
self.validate_identity(
"ALTER ALGORITHM = MERGE VIEW v AS SELECT * FROM foo", check_command_warning=True
)
self.validate_identity(
"ALTER DEFINER = 'admin'@'localhost' VIEW v AS SELECT * FROM foo",
check_command_warning=True,
)
self.validate_identity(
"ALTER SQL SECURITY = DEFINER VIEW v AS SELECT * FROM foo", check_command_warning=True
)
self.validate_identity(
"INSERT INTO things (a, b) VALUES (1, 2) AS new_data ON DUPLICATE KEY UPDATE id = LAST_INSERT_ID(id), a = new_data.a, b = new_data.b"
)

View file

@ -6,6 +6,7 @@ class TestOracle(Validator):
dialect = "oracle"
def test_oracle(self):
self.validate_identity("1 /* /* */")
self.validate_all(
"SELECT CONNECT_BY_ROOT x y",
write={
@ -13,8 +14,9 @@ class TestOracle(Validator):
"oracle": "SELECT CONNECT_BY_ROOT x AS y",
},
)
self.parse_one("ALTER TABLE tbl_name DROP FOREIGN KEY fk_symbol").assert_is(exp.AlterTable)
self.parse_one("ALTER TABLE tbl_name DROP FOREIGN KEY fk_symbol").assert_is(exp.Alter)
self.validate_identity("SYSDATE")
self.validate_identity("CREATE GLOBAL TEMPORARY TABLE t AS SELECT * FROM orders")
self.validate_identity("CREATE PRIVATE TEMPORARY TABLE t AS SELECT * FROM orders")
self.validate_identity("REGEXP_REPLACE('source', 'search')")
@ -43,6 +45,9 @@ class TestOracle(Validator):
self.validate_identity("SELECT COUNT(*) * 10 FROM orders SAMPLE (10) SEED (1)")
self.validate_identity("SELECT * FROM V$SESSION")
self.validate_identity("SELECT TO_DATE('January 15, 1989, 11:00 A.M.')")
self.validate_identity(
"SELECT * FROM test UNPIVOT INCLUDE NULLS (value FOR Description IN (col AS 'PREFIX ' || CHR(38) || ' SUFFIX'))"
)
self.validate_identity(
"SELECT last_name, employee_id, manager_id, LEVEL FROM employees START WITH employee_id = 100 CONNECT BY PRIOR employee_id = manager_id ORDER SIBLINGS BY last_name"
)
@ -72,10 +77,6 @@ class TestOracle(Validator):
"SELECT JSON_OBJECTAGG(KEY department_name VALUE department_id) FROM dep WHERE id <= 30",
"SELECT JSON_OBJECTAGG(department_name: department_id) FROM dep WHERE id <= 30",
)
self.validate_identity(
"SYSDATE",
"CURRENT_TIMESTAMP",
)
self.validate_identity(
"SELECT last_name, department_id, salary, MIN(salary) KEEP (DENSE_RANK FIRST ORDER BY commission_pct) "
'OVER (PARTITION BY department_id) AS "Worst", MAX(salary) KEEP (DENSE_RANK LAST ORDER BY commission_pct) '
@ -88,7 +89,6 @@ class TestOracle(Validator):
self.validate_identity(
"SELECT * FROM T ORDER BY I OFFSET NVL(:variable1, 10) ROWS FETCH NEXT NVL(:variable2, 10) ROWS ONLY",
)
self.validate_identity("NVL(x, y)").assert_is(exp.Anonymous)
self.validate_identity(
"SELECT * FROM t SAMPLE (.25)",
"SELECT * FROM t SAMPLE (0.25)",
@ -98,6 +98,16 @@ class TestOracle(Validator):
"SELECT * FROM t START WITH col CONNECT BY NOCYCLE PRIOR col1 = col2"
)
self.validate_all(
"SELECT * FROM test WHERE MOD(col1, 4) = 3",
read={
"duckdb": "SELECT * FROM test WHERE col1 % 4 = 3",
},
write={
"duckdb": "SELECT * FROM test WHERE col1 % 4 = 3",
"oracle": "SELECT * FROM test WHERE MOD(col1, 4) = 3",
},
)
self.validate_all(
"CURRENT_TIMESTAMP BETWEEN TO_DATE(f.C_SDATE, 'yyyy/mm/dd') AND TO_DATE(f.C_EDATE, 'yyyy/mm/dd')",
read={
@ -242,6 +252,15 @@ class TestOracle(Validator):
"""SELECT * FROM t ORDER BY a ASC, b ASC NULLS FIRST, c DESC NULLS LAST, d DESC""",
)
self.validate_all(
"NVL(NULL, 1)",
write={
"oracle": "NVL(NULL, 1)",
"": "COALESCE(NULL, 1)",
"clickhouse": "COALESCE(NULL, 1)",
},
)
def test_join_marker(self):
self.validate_identity("SELECT e1.x, e2.x FROM e e1, e e2 WHERE e1.y (+) = e2.y")

View file

@ -28,7 +28,7 @@ class TestPostgres(Validator):
alter_table_only = """ALTER TABLE ONLY "Album" ADD CONSTRAINT "FK_AlbumArtistId" FOREIGN KEY ("ArtistId") REFERENCES "Artist" ("ArtistId") ON DELETE NO ACTION ON UPDATE NO ACTION"""
expr = self.parse_one(alter_table_only)
self.assertIsInstance(expr, exp.AlterTable)
self.assertIsInstance(expr, exp.Alter)
self.assertEqual(expr.sql(dialect="postgres"), alter_table_only)
self.validate_identity("STRING_TO_ARRAY('xx~^~yy~^~zz', '~^~', 'yy')")
@ -548,47 +548,54 @@ class TestPostgres(Validator):
"postgres": "x # y",
},
)
self.validate_all(
"SELECT GENERATE_SERIES(1, 5)",
write={
"bigquery": UnsupportedError,
"postgres": "SELECT GENERATE_SERIES(1, 5)",
},
)
self.validate_all(
"WITH dates AS (SELECT GENERATE_SERIES('2020-01-01'::DATE, '2024-01-01'::DATE, '1 day'::INTERVAL) AS date), date_table AS (SELECT DISTINCT DATE_TRUNC('MONTH', date) AS date FROM dates) SELECT * FROM date_table",
write={
"duckdb": "WITH dates AS (SELECT UNNEST(GENERATE_SERIES(CAST('2020-01-01' AS DATE), CAST('2024-01-01' AS DATE), CAST('1 day' AS INTERVAL))) AS date), date_table AS (SELECT DISTINCT DATE_TRUNC('MONTH', date) AS date FROM dates) SELECT * FROM date_table",
"postgres": "WITH dates AS (SELECT GENERATE_SERIES(CAST('2020-01-01' AS DATE), CAST('2024-01-01' AS DATE), CAST('1 day' AS INTERVAL)) AS date), date_table AS (SELECT DISTINCT DATE_TRUNC('MONTH', date) AS date FROM dates) SELECT * FROM date_table",
},
)
self.validate_all(
"GENERATE_SERIES(a, b, ' 2 days ')",
write={
"postgres": "GENERATE_SERIES(a, b, INTERVAL '2 DAYS')",
"presto": "SEQUENCE(a, b, INTERVAL '2' DAY)",
"trino": "SEQUENCE(a, b, INTERVAL '2' DAY)",
"presto": "UNNEST(SEQUENCE(a, b, INTERVAL '2' DAY))",
"trino": "UNNEST(SEQUENCE(a, b, INTERVAL '2' DAY))",
},
)
self.validate_all(
"GENERATE_SERIES('2019-01-01'::TIMESTAMP, NOW(), '1day')",
write={
"databricks": "EXPLODE(SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY))",
"hive": "EXPLODE(SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY))",
"postgres": "GENERATE_SERIES(CAST('2019-01-01' AS TIMESTAMP), CURRENT_TIMESTAMP, INTERVAL '1 DAY')",
"presto": "SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP AS TIMESTAMP), INTERVAL '1' DAY)",
"trino": "SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP AS TIMESTAMP), INTERVAL '1' DAY)",
"hive": "SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY)",
"spark2": "SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY)",
"spark": "SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY)",
"databricks": "SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY)",
"presto": "UNNEST(SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP AS TIMESTAMP), INTERVAL '1' DAY))",
"spark": "EXPLODE(SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY))",
"spark2": "EXPLODE(SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP() AS TIMESTAMP), INTERVAL '1' DAY))",
"trino": "UNNEST(SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP AS TIMESTAMP), INTERVAL '1' DAY))",
},
)
self.validate_all(
"GENERATE_SERIES(a, b)",
"SELECT * FROM GENERATE_SERIES(a, b)",
read={
"postgres": "GENERATE_SERIES(a, b)",
"presto": "SEQUENCE(a, b)",
"trino": "SEQUENCE(a, b)",
"tsql": "GENERATE_SERIES(a, b)",
"hive": "SEQUENCE(a, b)",
"spark2": "SEQUENCE(a, b)",
"spark": "SEQUENCE(a, b)",
"databricks": "SEQUENCE(a, b)",
"tsql": "SELECT * FROM GENERATE_SERIES(a, b)",
},
write={
"postgres": "GENERATE_SERIES(a, b)",
"presto": "SEQUENCE(a, b)",
"trino": "SEQUENCE(a, b)",
"tsql": "GENERATE_SERIES(a, b)",
"hive": "SEQUENCE(a, b)",
"spark2": "SEQUENCE(a, b)",
"spark": "SEQUENCE(a, b)",
"databricks": "SEQUENCE(a, b)",
"databricks": "SELECT * FROM EXPLODE(SEQUENCE(a, b))",
"hive": "SELECT * FROM EXPLODE(SEQUENCE(a, b))",
"postgres": "SELECT * FROM GENERATE_SERIES(a, b)",
"presto": "SELECT * FROM UNNEST(SEQUENCE(a, b))",
"spark": "SELECT * FROM EXPLODE(SEQUENCE(a, b))",
"spark2": "SELECT * FROM EXPLODE(SEQUENCE(a, b))",
"trino": "SELECT * FROM UNNEST(SEQUENCE(a, b))",
"tsql": "SELECT * FROM GENERATE_SERIES(a, b)",
},
)
self.validate_all(

View file

@ -13,6 +13,13 @@ class TestPresto(Validator):
self.validate_identity("CAST(TDIGEST_AGG(1) AS TDIGEST)")
self.validate_identity("CAST(x AS HYPERLOGLOG)")
self.validate_all(
"SELECT FROM_ISO8601_TIMESTAMP('2020-05-11T11:15:05')",
write={
"duckdb": "SELECT CAST('2020-05-11T11:15:05' AS TIMESTAMPTZ)",
"presto": "SELECT FROM_ISO8601_TIMESTAMP('2020-05-11T11:15:05')",
},
)
self.validate_all(
"CAST(x AS INTERVAL YEAR TO MONTH)",
write={

View file

@ -6,7 +6,6 @@ class TestRedshift(Validator):
dialect = "redshift"
def test_redshift(self):
self.validate_identity("1 div", "1 AS div")
self.validate_all(
"SELECT SPLIT_TO_ARRAY('12,345,6789')",
write={
@ -315,6 +314,7 @@ class TestRedshift(Validator):
)
def test_identity(self):
self.validate_identity("1 div", "1 AS div")
self.validate_identity("LISTAGG(DISTINCT foo, ', ')")
self.validate_identity("CREATE MATERIALIZED VIEW orders AUTO REFRESH YES AS SELECT 1")
self.validate_identity("SELECT DATEADD(DAY, 1, 'today')")
@ -337,6 +337,10 @@ class TestRedshift(Validator):
self.validate_identity(
"""SELECT JSON_EXTRACT_PATH_TEXT('{"f2":{"f3":1},"f4":{"f5":99,"f6":"star"}', 'f4', 'f6', TRUE)"""
)
self.validate_identity(
'DATE_PART(year, "somecol")',
'EXTRACT(year FROM "somecol")',
).this.assert_is(exp.Var)
self.validate_identity(
"SELECT CONCAT('abc', 'def')",
"SELECT 'abc' || 'def'",
@ -430,6 +434,14 @@ ORDER BY
)
self.validate_identity("SELECT JSON_PARSE('[]')")
self.validate_identity("SELECT ARRAY(1, 2, 3)")
self.validate_identity("SELECT ARRAY[1, 2, 3]")
self.validate_identity(
"""SELECT CONVERT_TIMEZONE('America/New_York', '2024-08-06 09:10:00.000')""",
"""SELECT CONVERT_TIMEZONE('UTC', 'America/New_York', '2024-08-06 09:10:00.000')""",
)
def test_values(self):
# Test crazy-sized VALUES clause to UNION ALL conversion to ensure we don't get RecursionError
values = [str(v) for v in range(0, 10000)]
@ -608,3 +620,9 @@ FROM (
"select a.foo, b.bar, a.baz from a, b where a.baz = b.baz (+)",
"SELECT a.foo, b.bar, a.baz FROM a, b WHERE a.baz = b.baz (+)",
)
def test_time(self):
self.validate_all(
"TIME_TO_STR(a, '%Y-%m-%d %H:%M:%S.%f')",
write={"redshift": "TO_CHAR(a, 'YYYY-MM-DD HH24:MI:SS.US')"},
)

View file

@ -11,6 +11,7 @@ class TestSnowflake(Validator):
dialect = "snowflake"
def test_snowflake(self):
self.validate_identity("1 /* /* */")
self.validate_identity(
"SELECT * FROM table AT (TIMESTAMP => '2024-07-24') UNPIVOT(a FOR b IN (c)) AS pivot_table"
)
@ -59,6 +60,7 @@ WHERE
)""",
)
self.validate_identity("exclude := [foo]")
self.validate_identity("SELECT CAST([1, 2, 3] AS VECTOR(FLOAT, 3))")
self.validate_identity("SELECT CONNECT_BY_ROOT test AS test_column_alias")
self.validate_identity("SELECT number").selects[0].assert_is(exp.Column)
@ -113,6 +115,18 @@ WHERE
self.validate_identity("ALTER TABLE a SWAP WITH b")
self.validate_identity("SELECT MATCH_CONDITION")
self.validate_identity("SELECT * REPLACE (CAST(col AS TEXT) AS scol) FROM t")
self.validate_identity(
"SELECT * FROM quarterly_sales PIVOT(SUM(amount) FOR quarter IN ('2023_Q1', '2023_Q2', '2023_Q3', '2023_Q4', '2024_Q1') DEFAULT ON NULL (0)) ORDER BY empid"
)
self.validate_identity(
"SELECT * FROM quarterly_sales PIVOT(SUM(amount) FOR quarter IN (SELECT DISTINCT quarter FROM ad_campaign_types_by_quarter WHERE television = TRUE ORDER BY quarter)) ORDER BY empid"
)
self.validate_identity(
"SELECT * FROM quarterly_sales PIVOT(SUM(amount) FOR IN (ANY ORDER BY quarter)) ORDER BY empid"
)
self.validate_identity(
"SELECT * FROM quarterly_sales PIVOT(SUM(amount) FOR IN (ANY)) ORDER BY empid"
)
self.validate_identity(
"MERGE INTO my_db AS ids USING (SELECT new_id FROM my_model WHERE NOT col IS NULL) AS new_ids ON ids.type = new_ids.type AND ids.source = new_ids.source WHEN NOT MATCHED THEN INSERT VALUES (new_ids.new_id)"
)
@ -125,6 +139,18 @@ WHERE
self.validate_identity(
"SELECT * FROM DATA AS DATA_L ASOF JOIN DATA AS DATA_R MATCH_CONDITION (DATA_L.VAL > DATA_R.VAL) ON DATA_L.ID = DATA_R.ID"
)
self.validate_identity(
"SELECT * FROM s WHERE c NOT IN (1, 2, 3)",
"SELECT * FROM s WHERE NOT c IN (1, 2, 3)",
)
self.validate_identity(
"SELECT * FROM s WHERE c NOT IN (SELECT * FROM t)",
"SELECT * FROM s WHERE c <> ALL (SELECT * FROM t)",
)
self.validate_identity(
"SELECT * FROM t1 INNER JOIN t2 USING (t1.col)",
"SELECT * FROM t1 INNER JOIN t2 USING (col)",
)
self.validate_identity(
"CURRENT_TIMESTAMP - INTERVAL '1 w' AND (1 = 1)",
"CURRENT_TIMESTAMP() - INTERVAL '1 WEEK' AND (1 = 1)",
@ -847,6 +873,33 @@ WHERE
},
)
self.validate_identity(
"""SELECT ARRAY_CONSTRUCT('foo')::VARIANT[0]""",
"""SELECT CAST(['foo'] AS VARIANT)[0]""",
)
self.validate_all(
"SELECT CONVERT_TIMEZONE('America/New_York', '2024-08-06 09:10:00.000')",
write={
"snowflake": "SELECT CONVERT_TIMEZONE('America/New_York', '2024-08-06 09:10:00.000')",
"spark": "SELECT CONVERT_TIMEZONE('America/New_York', '2024-08-06 09:10:00.000')",
"databricks": "SELECT CONVERT_TIMEZONE('America/New_York', '2024-08-06 09:10:00.000')",
"redshift": "SELECT CONVERT_TIMEZONE('America/New_York', '2024-08-06 09:10:00.000')",
},
)
self.validate_all(
"SELECT CONVERT_TIMEZONE('America/Los_Angeles', 'America/New_York', '2024-08-06 09:10:00.000')",
write={
"snowflake": "SELECT CONVERT_TIMEZONE('America/Los_Angeles', 'America/New_York', '2024-08-06 09:10:00.000')",
"spark": "SELECT CONVERT_TIMEZONE('America/Los_Angeles', 'America/New_York', '2024-08-06 09:10:00.000')",
"databricks": "SELECT CONVERT_TIMEZONE('America/Los_Angeles', 'America/New_York', '2024-08-06 09:10:00.000')",
"redshift": "SELECT CONVERT_TIMEZONE('America/Los_Angeles', 'America/New_York', '2024-08-06 09:10:00.000')",
"mysql": "SELECT CONVERT_TZ('2024-08-06 09:10:00.000', 'America/Los_Angeles', 'America/New_York')",
"duckdb": "SELECT CAST('2024-08-06 09:10:00.000' AS TIMESTAMP) AT TIME ZONE 'America/Los_Angeles' AT TIME ZONE 'America/New_York'",
},
)
def test_null_treatment(self):
self.validate_all(
r"SELECT FIRST_VALUE(TABLE1.COLUMN1) OVER (PARTITION BY RANDOM_COLUMN1, RANDOM_COLUMN2 ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS MY_ALIAS FROM TABLE1",
@ -1921,7 +1974,7 @@ STORAGE_ALLOWED_LOCATIONS=('s3://mybucket1/path1/', 's3://mybucket2/path2/')""",
def test_swap(self):
ast = parse_one("ALTER TABLE a SWAP WITH b", read="snowflake")
assert isinstance(ast, exp.AlterTable)
assert isinstance(ast, exp.Alter)
assert isinstance(ast.args["actions"][0], exp.SwapTable)
def test_try_cast(self):

View file

@ -129,6 +129,16 @@ TBLPROPERTIES (
"spark": "ALTER TABLE StudentInfo DROP COLUMNS (LastName, DOB)",
},
)
self.validate_identity("ALTER VIEW StudentInfoView AS SELECT * FROM StudentInfo")
self.validate_identity("ALTER VIEW StudentInfoView AS SELECT LastName FROM StudentInfo")
self.validate_identity("ALTER VIEW StudentInfoView RENAME TO StudentInfoViewRenamed")
self.validate_identity(
"ALTER VIEW StudentInfoView SET TBLPROPERTIES ('key1'='val1', 'key2'='val2')"
)
self.validate_identity(
"ALTER VIEW StudentInfoView UNSET TBLPROPERTIES ('key1', 'key2')",
check_command_warning=True,
)
def test_to_date(self):
self.validate_all(
@ -297,6 +307,13 @@ TBLPROPERTIES (
},
)
self.validate_all(
"SELECT DATE_FORMAT(DATE '2020-01-01', 'EEEE') AS weekday",
write={
"presto": "SELECT DATE_FORMAT(CAST(CAST('2020-01-01' AS DATE) AS TIMESTAMP), '%W') AS weekday",
"spark": "SELECT DATE_FORMAT(CAST(CAST('2020-01-01' AS DATE) AS TIMESTAMP), 'EEEE') AS weekday",
},
)
self.validate_all(
"SELECT TRY_ELEMENT_AT(MAP(1, 'a', 2, 'b'), 2)",
read={
@ -557,7 +574,10 @@ TBLPROPERTIES (
)
self.validate_all(
"CAST(x AS TIMESTAMP)", read={"trino": "CAST(x AS TIMESTAMP(6) WITH TIME ZONE)"}
"CAST(x AS TIMESTAMP)",
read={
"trino": "CAST(x AS TIMESTAMP(6) WITH TIME ZONE)",
},
)
self.validate_all(
"SELECT DATE_ADD(my_date_column, 1)",
@ -688,6 +708,7 @@ TBLPROPERTIES (
"trino": "SELECT DATE_ADD('MONTH', 20, col)",
},
)
self.validate_identity("DESCRIBE schema.test PARTITION(ds = '2024-01-01')")
def test_bool_or(self):
self.validate_all(
@ -805,8 +826,22 @@ TBLPROPERTIES (
self.assertEqual(query.sql(name), without_modifiers)
def test_schema_binding_options(self):
for schema_binding in ("BINDING", "COMPENSATION", "TYPE EVOLUTION", "EVOLUTION"):
for schema_binding in (
"BINDING",
"COMPENSATION",
"TYPE EVOLUTION",
"EVOLUTION",
):
with self.subTest(f"Test roundtrip of VIEW schema binding {schema_binding}"):
self.validate_identity(
f"CREATE VIEW emp_v WITH SCHEMA {schema_binding} AS SELECT * FROM emp"
)
def test_minus(self):
self.validate_all(
"SELECT * FROM db.table1 MINUS SELECT * FROM db.table2",
write={
"spark": "SELECT * FROM db.table1 EXCEPT SELECT * FROM db.table2",
"databricks": "SELECT * FROM db.table1 EXCEPT SELECT * FROM db.table2",
},
)

View file

@ -1,3 +1,4 @@
from sqlglot import exp
from tests.dialects.test_dialect import Validator
@ -31,6 +32,10 @@ class TestTeradata(Validator):
},
)
self.validate_identity(
"RENAME TABLE emp TO employee", check_command_warning=True
).assert_is(exp.Command)
def test_translate(self):
self.validate_all(
"TRANSLATE(x USING LATIN_TO_UNICODE)",

View file

@ -16,3 +16,35 @@ class TestTrino(Validator):
"SELECT TRIM('!foo!', '!')",
"SELECT TRIM('!' FROM '!foo!')",
)
def test_ddl(self):
self.validate_identity("ALTER TABLE users RENAME TO people")
self.validate_identity("ALTER TABLE IF EXISTS users RENAME TO people")
self.validate_identity("ALTER TABLE users ADD COLUMN zip VARCHAR")
self.validate_identity("ALTER TABLE IF EXISTS users ADD COLUMN IF NOT EXISTS zip VARCHAR")
self.validate_identity("ALTER TABLE users DROP COLUMN zip")
self.validate_identity("ALTER TABLE IF EXISTS users DROP COLUMN IF EXISTS zip")
self.validate_identity("ALTER TABLE users RENAME COLUMN id TO user_id")
self.validate_identity("ALTER TABLE IF EXISTS users RENAME COLUMN IF EXISTS id TO user_id")
self.validate_identity("ALTER TABLE users ALTER COLUMN id SET DATA TYPE BIGINT")
self.validate_identity("ALTER TABLE users ALTER COLUMN id DROP NOT NULL")
self.validate_identity(
"ALTER TABLE people SET AUTHORIZATION alice", check_command_warning=True
)
self.validate_identity(
"ALTER TABLE people SET AUTHORIZATION ROLE PUBLIC", check_command_warning=True
)
self.validate_identity(
"ALTER TABLE people SET PROPERTIES x = 'y'", check_command_warning=True
)
self.validate_identity(
"ALTER TABLE people SET PROPERTIES foo = 123, 'foo bar' = 456",
check_command_warning=True,
)
self.validate_identity(
"ALTER TABLE people SET PROPERTIES x = DEFAULT", check_command_warning=True
)
self.validate_identity("ALTER VIEW people RENAME TO users")
self.validate_identity(
"ALTER VIEW people SET AUTHORIZATION alice", check_command_warning=True
)

View file

@ -792,7 +792,7 @@ class TestTSQL(Validator):
self.validate_identity(f"CREATE VIEW a.b WITH {view_attr} AS SELECT * FROM x")
self.validate_identity("ALTER TABLE dbo.DocExe DROP CONSTRAINT FK_Column_B").assert_is(
exp.AlterTable
exp.Alter
).args["actions"][0].assert_is(exp.Drop)
for clustered_keyword in ("CLUSTERED", "NONCLUSTERED"):
@ -822,6 +822,20 @@ class TestTSQL(Validator):
self.validate_identity("ALTER TABLE tbl SET DATA_DELETION=ON")
self.validate_identity("ALTER TABLE tbl SET DATA_DELETION=OFF")
self.validate_identity("ALTER VIEW v AS SELECT a, b, c, d FROM foo")
self.validate_identity("ALTER VIEW v AS SELECT * FROM foo WHERE c > 100")
self.validate_identity(
"ALTER VIEW v WITH SCHEMABINDING AS SELECT * FROM foo WHERE c > 100",
check_command_warning=True,
)
self.validate_identity(
"ALTER VIEW v WITH ENCRYPTION AS SELECT * FROM foo WHERE c > 100",
check_command_warning=True,
)
self.validate_identity(
"ALTER VIEW v WITH VIEW_METADATA AS SELECT * FROM foo WHERE c > 100",
check_command_warning=True,
)
self.validate_identity(
"CREATE PROCEDURE foo AS BEGIN DELETE FROM bla WHERE foo < CURRENT_TIMESTAMP - 7 END",
"CREATE PROCEDURE foo AS BEGIN DELETE FROM bla WHERE foo < GETDATE() - 7 END",
@ -1513,6 +1527,15 @@ WHERE
},
)
# Check superfluous casts arent added. ref: https://github.com/TobikoData/sqlmesh/issues/2672
self.validate_all(
"SELECT DATEDIFF(DAY, CAST(a AS DATETIME2), CAST(b AS DATETIME2)) AS x FROM foo",
write={
"tsql": "SELECT DATEDIFF(DAY, CAST(a AS DATETIME2), CAST(b AS DATETIME2)) AS x FROM foo",
"clickhouse": "SELECT DATE_DIFF(DAY, CAST(a AS Nullable(DateTime)), CAST(b AS Nullable(DateTime))) AS x FROM foo",
},
)
def test_lateral_subquery(self):
self.validate_all(
"SELECT x.a, x.b, t.v, t.y FROM x CROSS APPLY (SELECT v, y FROM t) t(v, y)",
@ -1650,7 +1673,7 @@ WHERE
},
write={
"bigquery": "LAST_DAY(CAST(CURRENT_TIMESTAMP() AS DATE))",
"clickhouse": "LAST_DAY(CAST(CURRENT_TIMESTAMP() AS DATE))",
"clickhouse": "LAST_DAY(CAST(CURRENT_TIMESTAMP() AS Nullable(DATE)))",
"duckdb": "LAST_DAY(CAST(CURRENT_TIMESTAMP AS DATE))",
"mysql": "LAST_DAY(DATE(CURRENT_TIMESTAMP()))",
"postgres": "CAST(DATE_TRUNC('MONTH', CAST(CURRENT_TIMESTAMP AS DATE)) + INTERVAL '1 MONTH' - INTERVAL '1 DAY' AS DATE)",
@ -1665,7 +1688,7 @@ WHERE
"EOMONTH(GETDATE(), -1)",
write={
"bigquery": "LAST_DAY(DATE_ADD(CAST(CURRENT_TIMESTAMP() AS DATE), INTERVAL -1 MONTH))",
"clickhouse": "LAST_DAY(DATE_ADD(MONTH, -1, CAST(CURRENT_TIMESTAMP() AS DATE)))",
"clickhouse": "LAST_DAY(DATE_ADD(MONTH, -1, CAST(CURRENT_TIMESTAMP() AS Nullable(DATE))))",
"duckdb": "LAST_DAY(CAST(CURRENT_TIMESTAMP AS DATE) + INTERVAL (-1) MONTH)",
"mysql": "LAST_DAY(DATE_ADD(CURRENT_TIMESTAMP(), INTERVAL -1 MONTH))",
"postgres": "CAST(DATE_TRUNC('MONTH', CAST(CURRENT_TIMESTAMP AS DATE) + INTERVAL '-1 MONTH') + INTERVAL '1 MONTH' - INTERVAL '1 DAY' AS DATE)",

Some files were not shown because too many files have changed in this diff Show more