python/sqlglot
1
0
Fork 0
Code Issues Activity

Adding upstream version 26.19.0.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-05-24 07:15:24 +02:00
parent 873e685933
commit 0eb0fedc25
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
98 changed files with 67345 additions and 65319 deletions
Download patch file Download diff file Expand all files Collapse all files

261
CHANGELOG.md
View file

@ -1,6 +1,265 @@
Changelog
=========
## [v26.18.1] - 2025-05-20
### :wrench: Chores
- [`db2af6f`](https://github.com/tobymao/sqlglot/commit/db2af6fa1e2c2bf0f4cebb272287d0b2e8e69f76) - bump sqlglotrs to 0.4.2 *(commit by [@georgesittas](https://github.com/georgesittas))*
## [v26.18.0] - 2025-05-20
### :boom: BREAKING CHANGES
- due to [`1df7f61`](https://github.com/tobymao/sqlglot/commit/1df7f611bc96616cb07950a80f6669d0bc331b0e) - refactor length_sql so it handles any type, not just varchar/blob *(PR [#4935](https://github.com/tobymao/sqlglot/pull/4935) by [@tekumara](https://github.com/tekumara))*:
refactor length_sql so it handles any type, not just varchar/blob (#4935)
- due to [`52719f3`](https://github.com/tobymao/sqlglot/commit/52719f37f6541e8ec9f66642ac23ed9015048092) - parse CREATE STAGE *(PR [#4947](https://github.com/tobymao/sqlglot/pull/4947) by [@tekumara](https://github.com/tekumara))*:
parse CREATE STAGE (#4947)
- due to [`fd39b30`](https://github.com/tobymao/sqlglot/commit/fd39b30209d068b787619b8137a105aca9c3e607) - parse CREATE FILE FORMAT *(PR [#4948](https://github.com/tobymao/sqlglot/pull/4948) by [@tekumara](https://github.com/tekumara))*:
parse CREATE FILE FORMAT (#4948)
- due to [`f835756`](https://github.com/tobymao/sqlglot/commit/f835756257f735643584b89e93693e8577744731) - Fix CREATE EXTERNAL TABLE properties *(PR [#4951](https://github.com/tobymao/sqlglot/pull/4951) by [@VaggelisD](https://github.com/VaggelisD))*:
Fix CREATE EXTERNAL TABLE properties (#4951)
- due to [`44b955b`](https://github.com/tobymao/sqlglot/commit/44b955bd537bfb8f5b6e84ecbcd5f6e3da852260) - Fix generation of exp.Values *(PR [#4930](https://github.com/tobymao/sqlglot/pull/4930) by [@VaggelisD](https://github.com/VaggelisD))*:
Fix generation of exp.Values (#4930)
- due to [`1f506b1`](https://github.com/tobymao/sqlglot/commit/1f506b186f1b954829195eefda318e231d474208) - support SHOW (ALL) TABLES *(PR [#4961](https://github.com/tobymao/sqlglot/pull/4961) by [@mscolnick](https://github.com/mscolnick))*:
support SHOW (ALL) TABLES (#4961)
- due to [`72cf4a4`](https://github.com/tobymao/sqlglot/commit/72cf4a4501a8d122041a28b71be5a41ffb53602a) - Add support for PIVOT multiple IN clauses *(PR [#4964](https://github.com/tobymao/sqlglot/pull/4964) by [@VaggelisD](https://github.com/VaggelisD))*:
Add support for PIVOT multiple IN clauses (#4964)
- due to [`400ea54`](https://github.com/tobymao/sqlglot/commit/400ea54d3a9cab256bfa5e496439bb9be6072d0b) - ensure JSON_FORMAT type is JSON when targeting Presto *(PR [#4968](https://github.com/tobymao/sqlglot/pull/4968) by [@georgesittas](https://github.com/georgesittas))*:
ensure JSON_FORMAT type is JSON when targeting Presto (#4968)
- due to [`cb20038`](https://github.com/tobymao/sqlglot/commit/cb2003875fc6e149bd4a631e99c312a04435a46b) - treat GO as command *(PR [#4978](https://github.com/tobymao/sqlglot/pull/4978) by [@georgesittas](https://github.com/georgesittas))*:
treat GO as command (#4978)
- due to [`60e26b8`](https://github.com/tobymao/sqlglot/commit/60e26b868242a05a7fdc2725bd21a127910a6fb7) - improve transpilability of GET_JSON_OBJECT by parsing json path *(PR [#4980](https://github.com/tobymao/sqlglot/pull/4980) by [@georgesittas](https://github.com/georgesittas))*:
improve transpilability of GET_JSON_OBJECT by parsing json path (#4980)
- due to [`2b7845a`](https://github.com/tobymao/sqlglot/commit/2b7845a3a821d366ae90ba9ef5e7d61194a34874) - Add support for Athena's Iceberg partitioning transforms *(PR [#4976](https://github.com/tobymao/sqlglot/pull/4976) by [@VaggelisD](https://github.com/VaggelisD))*:
Add support for Athena's Iceberg partitioning transforms (#4976)
- due to [`ee794e9`](https://github.com/tobymao/sqlglot/commit/ee794e9c6a3b2fdb142114327d904b6c94a16cd0) - use the standard POWER function instead of ^ fixes [#4982](https://github.com/tobymao/sqlglot/pull/4982) *(commit by [@georgesittas](https://github.com/georgesittas))*:
use the standard POWER function instead of ^ fixes #4982
- due to [`2369195`](https://github.com/tobymao/sqlglot/commit/2369195635e25dabd5ce26c13e402076508bba04) - consistently parse INTERVAL value as a string *(PR [#4986](https://github.com/tobymao/sqlglot/pull/4986) by [@georgesittas](https://github.com/georgesittas))*:
consistently parse INTERVAL value as a string (#4986)
- due to [`e866cff`](https://github.com/tobymao/sqlglot/commit/e866cffbaac3b62255d0d5c8be043ab2394af619) - support RELY option for PRIMARY KEY, FOREIGN KEY, and UNIQUE constraints *(PR [#4987](https://github.com/tobymao/sqlglot/pull/4987) by [@geooo109](https://github.com/geooo109))*:
support RELY option for PRIMARY KEY, FOREIGN KEY, and UNIQUE constraints (#4987)
- due to [`510984f`](https://github.com/tobymao/sqlglot/commit/510984f2ddc6ff13b8a8030f698aed9ad0e6f46b) - stop generating redundant TO_DATE calls *(PR [#4990](https://github.com/tobymao/sqlglot/pull/4990) by [@georgesittas](https://github.com/georgesittas))*:
stop generating redundant TO_DATE calls (#4990)
- due to [`da9ec61`](https://github.com/tobymao/sqlglot/commit/da9ec61e8edd5049e246390e1b638cf14d50fa2d) - Fix pretty generation of exp.Window *(PR [#4994](https://github.com/tobymao/sqlglot/pull/4994) by [@VaggelisD](https://github.com/VaggelisD))*:
Fix pretty generation of exp.Window (#4994)
- due to [`fb83fac`](https://github.com/tobymao/sqlglot/commit/fb83fac2d097d8d3e8e2556c072792857609bd94) - remove recursion from `simplify` *(PR [#4988](https://github.com/tobymao/sqlglot/pull/4988) by [@georgesittas](https://github.com/georgesittas))*:
remove recursion from `simplify` (#4988)
- due to [`890b24a`](https://github.com/tobymao/sqlglot/commit/890b24a5cec269f5595743d0a86024a23217a3f1) - remove `connector_depth` as it is now dead code *(commit by [@georgesittas](https://github.com/georgesittas))*:
remove `connector_depth` as it is now dead code
- due to [`1dc501b`](https://github.com/tobymao/sqlglot/commit/1dc501b8ed68638375d869e11f3bf188948a4990) - remove `max_depth` argument in simplify as it is now dead code *(commit by [@georgesittas](https://github.com/georgesittas))*:
remove `max_depth` argument in simplify as it is now dead code
- due to [`f5358d8`](https://github.com/tobymao/sqlglot/commit/f5358d8a3e2743b5ac0d540f10502d333ad4e082) - add support for GET statements *(PR [#5019](https://github.com/tobymao/sqlglot/pull/5019) by [@eruditmorina](https://github.com/eruditmorina))*:
add support for GET statements (#5019)
- due to [`bafa7f3`](https://github.com/tobymao/sqlglot/commit/bafa7f3a03c57e573b793ed2c83c3a549dfb789c) - parse DOW and DOY *(PR [#5037](https://github.com/tobymao/sqlglot/pull/5037) by [@geooo109](https://github.com/geooo109))*:
parse DOW and DOY (#5037)
- due to [`eb0a989`](https://github.com/tobymao/sqlglot/commit/eb0a989a7f3bbddb49c66ad5cd42043532568e25) - support udf environment property *(PR [#5045](https://github.com/tobymao/sqlglot/pull/5045) by [@geooo109](https://github.com/geooo109))*:
support udf environment property (#5045)
- due to [`807fbbc`](https://github.com/tobymao/sqlglot/commit/807fbbc5a89925fd3c98e823003a9dc929fcaff6) - transpile timestamp without time zone *(PR [#5047](https://github.com/tobymao/sqlglot/pull/5047) by [@geooo109](https://github.com/geooo109))*:
transpile timestamp without time zone (#5047)
- due to [`c48fc8f`](https://github.com/tobymao/sqlglot/commit/c48fc8fefc13becff92d0546cec1730f038af6b2) - support translate with error *(PR [#5052](https://github.com/tobymao/sqlglot/pull/5052) by [@geooo109](https://github.com/geooo109))*:
support translate with error (#5052)
- due to [`2e9704e`](https://github.com/tobymao/sqlglot/commit/2e9704ede255ef17b412c6905aad69afd70ccbf3) - Change `COLLATE` expression to `Var` for `ALTER TABLE` *(PR [#5055](https://github.com/tobymao/sqlglot/pull/5055) by [@MarcusRisanger](https://github.com/MarcusRisanger))*:
Change `COLLATE` expression to `Var` for `ALTER TABLE` (#5055)
- due to [`63f505e`](https://github.com/tobymao/sqlglot/commit/63f505e036928ed94df61a8b213bf84198e33d35) - unqualify UNNEST only the left most part of a column *(PR [#5069](https://github.com/tobymao/sqlglot/pull/5069) by [@geooo109](https://github.com/geooo109))*:
unqualify UNNEST only the left most part of a column (#5069)
- due to [`56da962`](https://github.com/tobymao/sqlglot/commit/56da9629899e72ab1e15cfc45ede838c4c38c16e) - to_timestamp without format *(PR [#5070](https://github.com/tobymao/sqlglot/pull/5070) by [@geooo109](https://github.com/geooo109))*:
to_timestamp without format (#5070)
- due to [`1ddfcbe`](https://github.com/tobymao/sqlglot/commit/1ddfcbe6c1d30d70533774da38d842bb3af6c205) - support CONVERT function *(PR [#5074](https://github.com/tobymao/sqlglot/pull/5074) by [@geooo109](https://github.com/geooo109))*:
support CONVERT function (#5074)
- due to [`ba52f01`](https://github.com/tobymao/sqlglot/commit/ba52f014f0d53ce8a179f1b140876274a01b38ac) - respect normalization strategy overrides *(PR [#5080](https://github.com/tobymao/sqlglot/pull/5080) by [@georgesittas](https://github.com/georgesittas))*:
respect normalization strategy overrides (#5080)
### :sparkles: New Features
- [`52719f3`](https://github.com/tobymao/sqlglot/commit/52719f37f6541e8ec9f66642ac23ed9015048092) - **snowflake**: parse CREATE STAGE *(PR [#4947](https://github.com/tobymao/sqlglot/pull/4947) by [@tekumara](https://github.com/tekumara))*
- [`fd39b30`](https://github.com/tobymao/sqlglot/commit/fd39b30209d068b787619b8137a105aca9c3e607) - **snowflake**: parse CREATE FILE FORMAT *(PR [#4948](https://github.com/tobymao/sqlglot/pull/4948) by [@tekumara](https://github.com/tekumara))*
- [`da9a6a1`](https://github.com/tobymao/sqlglot/commit/da9a6a1d56323319b87e9b193d12ad1c644b9239) - **snowflake**: parse SHOW STAGES *(PR [#4949](https://github.com/tobymao/sqlglot/pull/4949) by [@tekumara](https://github.com/tekumara))*
- [`bfdcdf0`](https://github.com/tobymao/sqlglot/commit/bfdcdf0afc0f4af3dacdfc3e8dca243793552b74) - **snowflake**: parse SHOW FILE FORMATS *(PR [#4950](https://github.com/tobymao/sqlglot/pull/4950) by [@tekumara](https://github.com/tekumara))*
- [`c591443`](https://github.com/tobymao/sqlglot/commit/c591443b6b2328780e08179144557e181db0cbb6) - **duckdb**: add support for GROUP clause in standard PIVOT syntax *(PR [#4953](https://github.com/tobymao/sqlglot/pull/4953) by [@georgesittas](https://github.com/georgesittas))*
- [`b011ee2`](https://github.com/tobymao/sqlglot/commit/b011ee2df0beaac75b982261a25d3e787dead54a) - **bigquery**: Add support for side & kind on set operators *(PR [#4959](https://github.com/tobymao/sqlglot/pull/4959) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *addresses issue [#4942](https://github.com/tobymao/sqlglot/issues/4942) opened by [@z3z1ma](https://github.com/z3z1ma)*
- [`1f506b1`](https://github.com/tobymao/sqlglot/commit/1f506b186f1b954829195eefda318e231d474208) - **duckdb**: support SHOW (ALL) TABLES *(PR [#4961](https://github.com/tobymao/sqlglot/pull/4961) by [@mscolnick](https://github.com/mscolnick))*
- :arrow_lower_right: *addresses issue [#4956](https://github.com/tobymao/sqlglot/issues/4956) opened by [@mscolnick](https://github.com/mscolnick)*
- [`ad5b595`](https://github.com/tobymao/sqlglot/commit/ad5b595049a16a27a7f249afea43dbcfcf43b5f4) - allow explicit aliasing in if(...) expressions *(PR [#4963](https://github.com/tobymao/sqlglot/pull/4963) by [@georgesittas](https://github.com/georgesittas))*
- [`72cf4a4`](https://github.com/tobymao/sqlglot/commit/72cf4a4501a8d122041a28b71be5a41ffb53602a) - **duckdb**: Add support for PIVOT multiple IN clauses *(PR [#4964](https://github.com/tobymao/sqlglot/pull/4964) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *addresses issue [#4944](https://github.com/tobymao/sqlglot/issues/4944) opened by [@nph](https://github.com/nph)*
- [`7bc5a21`](https://github.com/tobymao/sqlglot/commit/7bc5a217c3cc68d0cb1eaedc0c18f5188de80bf1) - **postgres**: support laterals with ordinality fixes [#4965](https://github.com/tobymao/sqlglot/pull/4965) *(PR [#4966](https://github.com/tobymao/sqlglot/pull/4966) by [@georgesittas](https://github.com/georgesittas))*
- [`400ea54`](https://github.com/tobymao/sqlglot/commit/400ea54d3a9cab256bfa5e496439bb9be6072d0b) - ensure JSON_FORMAT type is JSON when targeting Presto *(PR [#4968](https://github.com/tobymao/sqlglot/pull/4968) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#4967](https://github.com/tobymao/sqlglot/issues/4967) opened by [@jmsmdy](https://github.com/jmsmdy)*
- [`a762993`](https://github.com/tobymao/sqlglot/commit/a762993c53d7ae91a831a8be448010e17e60f497) - **generator**: unsupported warning for T-SQL query option *(PR [#4972](https://github.com/tobymao/sqlglot/pull/4972) by [@geooo109](https://github.com/geooo109))*
- [`e866cff`](https://github.com/tobymao/sqlglot/commit/e866cffbaac3b62255d0d5c8be043ab2394af619) - **parser**: support RELY option for PRIMARY KEY, FOREIGN KEY, and UNIQUE constraints *(PR [#4987](https://github.com/tobymao/sqlglot/pull/4987) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *addresses issue [#4983](https://github.com/tobymao/sqlglot/issues/4983) opened by [@ggadon](https://github.com/ggadon)*
- [`76535ce`](https://github.com/tobymao/sqlglot/commit/76535ce9487186d2eb7071fac2f224238de7a9ba) - **optimizer**: add support for Spark's TRANSFORM clause *(PR [#4993](https://github.com/tobymao/sqlglot/pull/4993) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#4991](https://github.com/tobymao/sqlglot/issues/4991) opened by [@karta0807913](https://github.com/karta0807913)*
- [`27a9fb2`](https://github.com/tobymao/sqlglot/commit/27a9fb26a1936512a09b8b09ed2656e22918f2c6) - **clickhouse**: Support parsing CTAS with alias *(PR [#5003](https://github.com/tobymao/sqlglot/pull/5003) by [@dorranh](https://github.com/dorranh))*
- [`45cd165`](https://github.com/tobymao/sqlglot/commit/45cd165eaca96b33f1de753a147bdc352b9d56d0) - **clickhouse**: Support ClickHouse Nothing type *(PR [#5004](https://github.com/tobymao/sqlglot/pull/5004) by [@dorranh](https://github.com/dorranh))*
- [`ca61a61`](https://github.com/tobymao/sqlglot/commit/ca61a617fa67082bc0fc94853dee4d70b8ca5c59) - Support exp.PartitionByProperty for parse_into() *(PR [#5006](https://github.com/tobymao/sqlglot/pull/5006) by [@erindru](https://github.com/erindru))*
- [`a6d4c3c`](https://github.com/tobymao/sqlglot/commit/a6d4c3c901f828cdd96a16a0e55eac1b244f63be) - **snowflake**: Add numeric parameter support *(PR [#5008](https://github.com/tobymao/sqlglot/pull/5008) by [@hovaesco](https://github.com/hovaesco))*
- [`5feae00`](https://github.com/tobymao/sqlglot/commit/5feae00ec7a4826285e7fd0be85d377cc0de09b5) - **databricks**: add support for the VOID type *(PR [#5012](https://github.com/tobymao/sqlglot/pull/5012) by [@georgesittas](https://github.com/georgesittas))*
- [`6010302`](https://github.com/tobymao/sqlglot/commit/60103020879db5f23a6c4a1775848e31cce13415) - **postgres**: transpile QUARTER interval unit *(PR [#5015](https://github.com/tobymao/sqlglot/pull/5015) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#5013](https://github.com/tobymao/sqlglot/issues/5013) opened by [@Wiill007](https://github.com/Wiill007)*
- [`f5358d8`](https://github.com/tobymao/sqlglot/commit/f5358d8a3e2743b5ac0d540f10502d333ad4e082) - **snowflake**: add support for GET statements *(PR [#5019](https://github.com/tobymao/sqlglot/pull/5019) by [@eruditmorina](https://github.com/eruditmorina))*
- [`df5ecdb`](https://github.com/tobymao/sqlglot/commit/df5ecdbebcdce491031538f6baa0f87ec7eefee8) - Include token refereces in the meta of identifier expressions *(PR [#5022](https://github.com/tobymao/sqlglot/pull/5022) by [@izeigerman](https://github.com/izeigerman))*
- [`bafa7f3`](https://github.com/tobymao/sqlglot/commit/bafa7f3a03c57e573b793ed2c83c3a549dfb789c) - **presto**: parse DOW and DOY *(PR [#5037](https://github.com/tobymao/sqlglot/pull/5037) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *addresses issue [#5036](https://github.com/tobymao/sqlglot/issues/5036) opened by [@baruchoxman](https://github.com/baruchoxman)*
- [`eb0a989`](https://github.com/tobymao/sqlglot/commit/eb0a989a7f3bbddb49c66ad5cd42043532568e25) - support udf environment property *(PR [#5045](https://github.com/tobymao/sqlglot/pull/5045) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *addresses issue [#5043](https://github.com/tobymao/sqlglot/issues/5043) opened by [@aersam](https://github.com/aersam)*
- [`c48fc8f`](https://github.com/tobymao/sqlglot/commit/c48fc8fefc13becff92d0546cec1730f038af6b2) - **teradata**: support translate with error *(PR [#5052](https://github.com/tobymao/sqlglot/pull/5052) by [@geooo109](https://github.com/geooo109))*
- [`6791849`](https://github.com/tobymao/sqlglot/commit/679184943f7ffa79a2a466546f9bdfccd69034a3) - **executor**: support conversion from table to pylist *(PR [#5053](https://github.com/tobymao/sqlglot/pull/5053) by [@esadek](https://github.com/esadek))*
- [`07bf71b`](https://github.com/tobymao/sqlglot/commit/07bf71bae5d2a5c381104a86bb52c06809c21174) - **parser**: FK REFERENCES without specifying column *(PR [#5064](https://github.com/tobymao/sqlglot/pull/5064) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *addresses issue [#5057](https://github.com/tobymao/sqlglot/issues/5057) opened by [@Steven-Wright](https://github.com/Steven-Wright)*
- [`1ddfcbe`](https://github.com/tobymao/sqlglot/commit/1ddfcbe6c1d30d70533774da38d842bb3af6c205) - **oracle**: support CONVERT function *(PR [#5074](https://github.com/tobymao/sqlglot/pull/5074) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *addresses issue [#5071](https://github.com/tobymao/sqlglot/issues/5071) opened by [@tchamwam](https://github.com/tchamwam)*
- [`2cca655`](https://github.com/tobymao/sqlglot/commit/2cca655430ccf4542dcb3fd0e95b776739ef91eb) - allow PIVOT to follow a JOIN *(PR [#5075](https://github.com/tobymao/sqlglot/pull/5075) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#5073](https://github.com/tobymao/sqlglot/issues/5073) opened by [@tchamwam](https://github.com/tchamwam)*
- [`c7a56d7`](https://github.com/tobymao/sqlglot/commit/c7a56d7616cfb99de942d527e80ccec36cfc5cc3) - **oracle**: PRIOR in SELECT *(PR [#5077](https://github.com/tobymao/sqlglot/pull/5077) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *addresses issue [#5072](https://github.com/tobymao/sqlglot/issues/5072) opened by [@tchamwam](https://github.com/tchamwam)*
- [`5c66679`](https://github.com/tobymao/sqlglot/commit/5c66679208b34b480b9a0a0c538a15ab98f872b6) - **clickhouse**: allow EXCHANGE to be parsed as Command *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`3bcf989`](https://github.com/tobymao/sqlglot/commit/3bcf9899bbdac54bf8923ab3aa13ec66c65f0c44) - **snowflake**: Transpile DataType.BIGDECIMAL to DOUBLE *(PR [#5092](https://github.com/tobymao/sqlglot/pull/5092) by [@VaggelisD](https://github.com/VaggelisD))*
- [`b63b60e`](https://github.com/tobymao/sqlglot/commit/b63b60ebd10ca51f05e3f54532767bd98ccc34e3) - treat `CHAR[ACTER] VARYING` as `VARCHAR` for all dialects *(PR [#5093](https://github.com/tobymao/sqlglot/pull/5093) by [@ewhitley](https://github.com/ewhitley))*
- [`aa26aad`](https://github.com/tobymao/sqlglot/commit/aa26aad2608cd55b8bbd1d9e268444307a7224dc) - transpile WINDOW clause *(PR [#5097](https://github.com/tobymao/sqlglot/pull/5097) by [@georgesittas](https://github.com/georgesittas))*
### :bug: Bug Fixes
- [`7042603`](https://github.com/tobymao/sqlglot/commit/7042603ecb5693795b15219ec9cebf2f76032c03) - **optimizer**: Merge subqueries when inner query has name conflict with outer query *(PR [#4931](https://github.com/tobymao/sqlglot/pull/4931) by [@barakalon](https://github.com/barakalon))*
- [`1df7f61`](https://github.com/tobymao/sqlglot/commit/1df7f611bc96616cb07950a80f6669d0bc331b0e) - **duckdb**: refactor length_sql so it handles any type, not just varchar/blob *(PR [#4935](https://github.com/tobymao/sqlglot/pull/4935) by [@tekumara](https://github.com/tekumara))*
- :arrow_lower_right: *fixes issue [#4934](https://github.com/tobymao/sqlglot/issues/4934) opened by [@tekumara](https://github.com/tekumara)*
- [`09882e3`](https://github.com/tobymao/sqlglot/commit/09882e32f057670a9cbd97c1e5cf1a00c774b5d2) - **tsql**: remove assert call from _build_formatted_time *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`bf39a95`](https://github.com/tobymao/sqlglot/commit/bf39a95426ed6637e424da1be070cc9a8affc358) - **sqlite**: transpile double quoted PRIMARY KEY *(PR [#4941](https://github.com/tobymao/sqlglot/pull/4941) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *fixes issue [#4938](https://github.com/tobymao/sqlglot/issues/4938) opened by [@rgeronimi](https://github.com/rgeronimi)*
- [`f835756`](https://github.com/tobymao/sqlglot/commit/f835756257f735643584b89e93693e8577744731) - **snowflake**: Fix CREATE EXTERNAL TABLE properties *(PR [#4951](https://github.com/tobymao/sqlglot/pull/4951) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4945](https://github.com/tobymao/sqlglot/issues/4945) opened by [@tekumara](https://github.com/tekumara)*
- [`61ed971`](https://github.com/tobymao/sqlglot/commit/61ed971213c979c3777e57853bd6989bc169adb1) - **athena**: Correctly handle CTAS queries that contain Union's *(PR [#4955](https://github.com/tobymao/sqlglot/pull/4955) by [@erindru](https://github.com/erindru))*
- [`44b955b`](https://github.com/tobymao/sqlglot/commit/44b955bd537bfb8f5b6e84ecbcd5f6e3da852260) - **clickhouse**: Fix generation of exp.Values *(PR [#4930](https://github.com/tobymao/sqlglot/pull/4930) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4056](https://github.com/TobikoData/sqlmesh/issues/4056) opened by [@dnbnero](https://github.com/dnbnero)*
- [`61bc01c`](https://github.com/tobymao/sqlglot/commit/61bc01ceec2f801490f3f1a571aee655c5109962) - **clickhouse**: allow string literal for clickhouse ON CLUSTER clause *(PR [#4971](https://github.com/tobymao/sqlglot/pull/4971) by [@lepfhty](https://github.com/lepfhty))*
- [`1353b79`](https://github.com/tobymao/sqlglot/commit/1353b79bd9810788a02163928b044fe038267078) - **Snowflake**: Enhance parity for FILE_FORMAT & CREDENTIALS in CREATE STAGE *(PR [#4969](https://github.com/tobymao/sqlglot/pull/4969) by [@whummer](https://github.com/whummer))*
- [`9693dbd`](https://github.com/tobymao/sqlglot/commit/9693dbd18b98b2699cade738a254f71f2ee8ce74) - **clickhouse**: avoid superfluous parentheses in DISTINCT ON (...) *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`cb20038`](https://github.com/tobymao/sqlglot/commit/cb2003875fc6e149bd4a631e99c312a04435a46b) - **tsql**: treat GO as command *(PR [#4978](https://github.com/tobymao/sqlglot/pull/4978) by [@georgesittas](https://github.com/georgesittas))*
- [`60e26b8`](https://github.com/tobymao/sqlglot/commit/60e26b868242a05a7fdc2725bd21a127910a6fb7) - **hive**: improve transpilability of GET_JSON_OBJECT by parsing json path *(PR [#4980](https://github.com/tobymao/sqlglot/pull/4980) by [@georgesittas](https://github.com/georgesittas))*
- [`2b7845a`](https://github.com/tobymao/sqlglot/commit/2b7845a3a821d366ae90ba9ef5e7d61194a34874) - Add support for Athena's Iceberg partitioning transforms *(PR [#4976](https://github.com/tobymao/sqlglot/pull/4976) by [@VaggelisD](https://github.com/VaggelisD))*
- [`fa6af23`](https://github.com/tobymao/sqlglot/commit/fa6af2302f8482c5d89ead481afe4195aaa41a9c) - **optimizer**: compare the whole type to determine if a cast can be removed *(PR [#4981](https://github.com/tobymao/sqlglot/pull/4981) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#4977](https://github.com/tobymao/sqlglot/issues/4977) opened by [@MeinAccount](https://github.com/MeinAccount)*
- [`830c9b8`](https://github.com/tobymao/sqlglot/commit/830c9b8bbf906cf5d4fa8028b67dadda73fc58a9) - **unnest_subqueries**: avoid adding GROUP BY on aggregate projections in lateral subqueries *(PR [#4970](https://github.com/tobymao/sqlglot/pull/4970) by [@skadel](https://github.com/skadel))*
- [`ee794e9`](https://github.com/tobymao/sqlglot/commit/ee794e9c6a3b2fdb142114327d904b6c94a16cd0) - **postgres**: use the standard POWER function instead of ^ fixes [#4982](https://github.com/tobymao/sqlglot/pull/4982) *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`85e62b8`](https://github.com/tobymao/sqlglot/commit/85e62b88df2822797f527dce4eaa230c778cbe9e) - **bigquery**: Do not consume JOIN keywords after WITH OFFSET *(PR [#4984](https://github.com/tobymao/sqlglot/pull/4984) by [@VaggelisD](https://github.com/VaggelisD))*
- [`2369195`](https://github.com/tobymao/sqlglot/commit/2369195635e25dabd5ce26c13e402076508bba04) - consistently parse INTERVAL value as a string *(PR [#4986](https://github.com/tobymao/sqlglot/pull/4986) by [@georgesittas](https://github.com/georgesittas))*
- [`510984f`](https://github.com/tobymao/sqlglot/commit/510984f2ddc6ff13b8a8030f698aed9ad0e6f46b) - **hive**: stop generating redundant TO_DATE calls *(PR [#4990](https://github.com/tobymao/sqlglot/pull/4990) by [@georgesittas](https://github.com/georgesittas))*
- [`da9ec61`](https://github.com/tobymao/sqlglot/commit/da9ec61e8edd5049e246390e1b638cf14d50fa2d) - **generator**: Fix pretty generation of exp.Window *(PR [#4994](https://github.com/tobymao/sqlglot/pull/4994) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4098](https://github.com/TobikoData/sqlmesh/issues/4098) opened by [@tanghyd](https://github.com/tanghyd)*
- [`aae9aa8`](https://github.com/tobymao/sqlglot/commit/aae9aa8f96ccaa7686cda3cdabec208ae4c3d60a) - **optimizer**: ensure there are no shared refs after qualify_tables *(PR [#4995](https://github.com/tobymao/sqlglot/pull/4995) by [@georgesittas](https://github.com/georgesittas))*
- [`adaef42`](https://github.com/tobymao/sqlglot/commit/adaef42234d8f1c9c331f53bee2c42686f29bdec) - **trino**: Dont quote identifiers in string literals for the partitioned_by property *(PR [#4998](https://github.com/tobymao/sqlglot/pull/4998) by [@erindru](https://github.com/erindru))*
- [`a547f8d`](https://github.com/tobymao/sqlglot/commit/a547f8d4292f3b3a4c85f9d6466ead2ad976dfd2) - **postgres**: Capture optional minus sign in interval regex *(PR [#5000](https://github.com/tobymao/sqlglot/pull/5000) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#4999](https://github.com/tobymao/sqlglot/issues/4999) opened by [@cpimhoff](https://github.com/cpimhoff)*
- [`8e9dbd4`](https://github.com/tobymao/sqlglot/commit/8e9dbd491b9516c614554e05f05cc1cb976838e3) - **duckdb**: warn on unsupported IGNORE/RESPECT NULLS *(PR [#5002](https://github.com/tobymao/sqlglot/pull/5002) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#5001](https://github.com/tobymao/sqlglot/issues/5001) opened by [@MarcoGorelli](https://github.com/MarcoGorelli)*
- [`10b02bc`](https://github.com/tobymao/sqlglot/commit/10b02bce304042fea09e9cb2369db3c873452245) - **clickhouse**: Support optional timezone argument in date_diff() *(PR [#5005](https://github.com/tobymao/sqlglot/pull/5005) by [@dorranh](https://github.com/dorranh))*
- [`c594b63`](https://github.com/tobymao/sqlglot/commit/c594b630c1c940e9a47abfce1633b435a2607f13) - Add MAX_BY & MIN_BY to FUNCTION_PARSER *(PR [#5021](https://github.com/tobymao/sqlglot/pull/5021) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#5020](https://github.com/tobymao/sqlglot/issues/5020) opened by [@omerhadari](https://github.com/omerhadari)*
- [`c1c892c`](https://github.com/tobymao/sqlglot/commit/c1c892cebb89ddf29369ff3c7647f96d217acb71) - **parser**: parse column ops after no-paren type casting *(PR [#5025](https://github.com/tobymao/sqlglot/pull/5025) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#5024](https://github.com/tobymao/sqlglot/issues/5024) opened by [@MagdaSousa](https://github.com/MagdaSousa)*
- [`52e068f`](https://github.com/tobymao/sqlglot/commit/52e068f74bd6844d0273ddcc7637d249e6ed51c1) - **databricks**: Preserve colon operators in TRY_CAST *(PR [#5028](https://github.com/tobymao/sqlglot/pull/5028) by [@VaggelisD](https://github.com/VaggelisD))*
- :arrow_lower_right: *fixes issue [#5027](https://github.com/tobymao/sqlglot/issues/5027) opened by [@aersam](https://github.com/aersam)*
- [`91e5036`](https://github.com/tobymao/sqlglot/commit/91e5036831b87fd4670424e6a49e81efead432f2) - **parser**: Do not parse set ops if input expr is None *(PR [#5030](https://github.com/tobymao/sqlglot/pull/5030) by [@VaggelisD](https://github.com/VaggelisD))*
- [`8f77b30`](https://github.com/tobymao/sqlglot/commit/8f77b301a267eadb4c4792201e112159db554d1c) - **snowflake**: get function *(commit by [@tobymao](https://github.com/tobymao))*
- [`281ab21`](https://github.com/tobymao/sqlglot/commit/281ab21969d3937cef55adc3032f74b00173e948) - **snowflake**: generate expression DayOfWeekIso using DAYOFWEEKISO *(PR [#5034](https://github.com/tobymao/sqlglot/pull/5034) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *fixes issue [#5032](https://github.com/tobymao/sqlglot/issues/5032) opened by [@baruchoxman](https://github.com/baruchoxman)*
- [`2fa9684`](https://github.com/tobymao/sqlglot/commit/2fa96843a29323b97229842f7cf993b72bc86677) - preserve non-participating joins in eliminate_join_marks rule fixes [#5039](https://github.com/tobymao/sqlglot/pull/5039) *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`d10fdf5`](https://github.com/tobymao/sqlglot/commit/d10fdf5f9388dc3848617cfbf4e6f7b1aa73be1a) - **optimizer**: prevent incorrect predicate pushdown into RHS of CROSS JOIN UNNEST *(PR [#5033](https://github.com/tobymao/sqlglot/pull/5033) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *fixes issue [#5023](https://github.com/tobymao/sqlglot/issues/5023) opened by [@schelip](https://github.com/schelip)*
- [`7c55c48`](https://github.com/tobymao/sqlglot/commit/7c55c48ec2088e776fd4ec5b6c0f4989450a39c6) - prevent redundant backslash escapes in rawstring generator *(PR [#5040](https://github.com/tobymao/sqlglot/pull/5040) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#5038](https://github.com/tobymao/sqlglot/issues/5038) opened by [@ihvol-freenome](https://github.com/ihvol-freenome)*
- [`167c547`](https://github.com/tobymao/sqlglot/commit/167c547171fa3f2de1c2fdd64ca51bb9ccb3ee52) - **tsql**: ALTER COLUMN syntax *(PR [#5051](https://github.com/tobymao/sqlglot/pull/5051) by [@MarcusRisanger](https://github.com/MarcusRisanger))*
- :arrow_lower_right: *fixes issue [#5050](https://github.com/tobymao/sqlglot/issues/5050) opened by [@MarcusRisanger](https://github.com/MarcusRisanger)*
- [`807fbbc`](https://github.com/tobymao/sqlglot/commit/807fbbc5a89925fd3c98e823003a9dc929fcaff6) - **duckdb**: transpile timestamp without time zone *(PR [#5047](https://github.com/tobymao/sqlglot/pull/5047) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *fixes issue [#4859](https://github.com/tobymao/sqlglot/issues/4859) opened by [@eakmanrq](https://github.com/eakmanrq)*
- [`2e9704e`](https://github.com/tobymao/sqlglot/commit/2e9704ede255ef17b412c6905aad69afd70ccbf3) - **tsql**: Change `COLLATE` expression to `Var` for `ALTER TABLE` *(PR [#5055](https://github.com/tobymao/sqlglot/pull/5055) by [@MarcusRisanger](https://github.com/MarcusRisanger))*
- [`60f9420`](https://github.com/tobymao/sqlglot/commit/60f9420660d8d48bd98560a9bf8aec1f497fdeff) - **druid**: preserve MOD function fixes [#5060](https://github.com/tobymao/sqlglot/pull/5060) *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`7866b48`](https://github.com/tobymao/sqlglot/commit/7866b48275c830aeb51592e1888c751bcb58a361) - **druid**: support current_timestamp *(PR [#5061](https://github.com/tobymao/sqlglot/pull/5061) by [@ALongJohnson](https://github.com/ALongJohnson))*
- :arrow_lower_right: *fixes issue [#5059](https://github.com/tobymao/sqlglot/issues/5059) opened by [@ALongJohnson](https://github.com/ALongJohnson)*
- [`626f3a3`](https://github.com/tobymao/sqlglot/commit/626f3a3987c2a96a8fd6e329d237c0c7bc8bf264) - Support EXCLUDE in window definition *(PR [#5058](https://github.com/tobymao/sqlglot/pull/5058) by [@rafasofizada](https://github.com/rafasofizada))*
- [`63f505e`](https://github.com/tobymao/sqlglot/commit/63f505e036928ed94df61a8b213bf84198e33d35) - unqualify UNNEST only the left most part of a column *(PR [#5069](https://github.com/tobymao/sqlglot/pull/5069) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *fixes issue [#5062](https://github.com/tobymao/sqlglot/issues/5062) opened by [@goldmedal](https://github.com/goldmedal)*
- [`56da962`](https://github.com/tobymao/sqlglot/commit/56da9629899e72ab1e15cfc45ede838c4c38c16e) - **oracle**: to_timestamp without format *(PR [#5070](https://github.com/tobymao/sqlglot/pull/5070) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *fixes issue [#5068](https://github.com/tobymao/sqlglot/issues/5068) opened by [@kosta-foundational](https://github.com/kosta-foundational)*
- [`ba52f01`](https://github.com/tobymao/sqlglot/commit/ba52f014f0d53ce8a179f1b140876274a01b38ac) - **bigquery**: respect normalization strategy overrides *(PR [#5080](https://github.com/tobymao/sqlglot/pull/5080) by [@georgesittas](https://github.com/georgesittas))*
- [`03ace87`](https://github.com/tobymao/sqlglot/commit/03ace877e3f9e5d56fcbcbe260849f5d1247e5d9) - **optimizer**: keep ORDER BY when merging subqueries *(PR [#5084](https://github.com/tobymao/sqlglot/pull/5084) by [@geooo109](https://github.com/geooo109))*
- :arrow_lower_right: *fixes issue [#5065](https://github.com/tobymao/sqlglot/issues/5065) opened by [@udaykrishna-eng](https://github.com/udaykrishna-eng)*
- [`ba7b5a8`](https://github.com/tobymao/sqlglot/commit/ba7b5a8566dc15f438dcd0c03397b2e93e9c75cb) - **bigquery**: respect normalization strategy patching *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`4558bb7`](https://github.com/tobymao/sqlglot/commit/4558bb7a3a00629194f969d05d4b151f9ccd6172) - **bigquery**: always infer concat type as either bytes or string *(PR [#5085](https://github.com/tobymao/sqlglot/pull/5085) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *fixes issue [#5083](https://github.com/tobymao/sqlglot/issues/5083) opened by [@tobymao](https://github.com/tobymao)*
- [`612a2da`](https://github.com/tobymao/sqlglot/commit/612a2daeb0e93c5cc77b3c78c0b53905f4bee19c) - **tokenizer**: fix token col attribute when there is leading whitespace after a newline *(PR [#5094](https://github.com/tobymao/sqlglot/pull/5094) by [@chgiff](https://github.com/chgiff))*
- [`9d3a929`](https://github.com/tobymao/sqlglot/commit/9d3a929ba9006ebac67ff315c55da74a724ec975) - preserve `ARRAY_JOIN` for StarRocks, Doris (fixes [#5095](https://github.com/tobymao/sqlglot/pull/5095)) *(commit by [@georgesittas](https://github.com/georgesittas))*
### :recycle: Refactors
- [`fb83fac`](https://github.com/tobymao/sqlglot/commit/fb83fac2d097d8d3e8e2556c072792857609bd94) - **optimizer**: remove recursion from `simplify` *(PR [#4988](https://github.com/tobymao/sqlglot/pull/4988) by [@georgesittas](https://github.com/georgesittas))*
- [`1b3ea34`](https://github.com/tobymao/sqlglot/commit/1b3ea344af1d71d3eee239a5c4996a0aecd091de) - **clickhouse**: override _parse_property_assignment to handle null engine *(commit by [@georgesittas](https://github.com/georgesittas))*
### :wrench: Chores
- [`890b24a`](https://github.com/tobymao/sqlglot/commit/890b24a5cec269f5595743d0a86024a23217a3f1) - remove `connector_depth` as it is now dead code *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`1dc501b`](https://github.com/tobymao/sqlglot/commit/1dc501b8ed68638375d869e11f3bf188948a4990) - remove `max_depth` argument in simplify as it is now dead code *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`6572517`](https://github.com/tobymao/sqlglot/commit/6572517c1ec76f14cbd661aacc15c84bef065284) - improve tooling around benchmarks *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`1d4d906`](https://github.com/tobymao/sqlglot/commit/1d4d906abc60d29b6606bc8eee50c92cef21d3fd) - use _try_parse for parsing ClickHouse's CREATE TABLE .. AS <table> *(commit by [@georgesittas](https://github.com/georgesittas))*
- [`fc58c27`](https://github.com/tobymao/sqlglot/commit/fc58c273690734263b971b138ec8f0186f524672) - Refactor placeholder parsing for TokenType.COLON *(PR [#5009](https://github.com/tobymao/sqlglot/pull/5009) by [@VaggelisD](https://github.com/VaggelisD))*
- [`da90228`](https://github.com/tobymao/sqlglot/commit/da90228f1550715646106dd6f9a170d0973f138f) - put a lock around the lazy dialect module loading call *(PR [#5011](https://github.com/tobymao/sqlglot/pull/5011) by [@georgesittas](https://github.com/georgesittas))*
- :arrow_lower_right: *addresses issue [#5010](https://github.com/tobymao/sqlglot/issues/5010) opened by [@NickCrews](https://github.com/NickCrews)*
- [`abbcf26`](https://github.com/tobymao/sqlglot/commit/abbcf26b2101b2d806466353dcd29b79d1af5219) - bump sqlglotrs to 0.4.1 *(commit by [@georgesittas](https://github.com/georgesittas))*
## [v26.16.4] - 2025-05-02
### :bug: Bug Fixes
- [`52e068f`](https://github.com/tobymao/sqlglot/commit/52e068f74bd6844d0273ddcc7637d249e6ed51c1) - **databricks**: Preserve colon operators in TRY_CAST *(PR [#5028](https://github.com/tobymao/sqlglot/pull/5028) by [@VaggelisD](https://github.com/VaggelisD))*
@ -6496,3 +6755,5 @@ Changelog
[v26.16.2]: https://github.com/tobymao/sqlglot/compare/v26.16.1...v26.16.2
[v26.16.3]: https://github.com/tobymao/sqlglot/compare/v26.16.2...v26.16.3
[v26.16.4]: https://github.com/tobymao/sqlglot/compare/v26.16.3...v26.16.4
[v26.18.0]: https://github.com/tobymao/sqlglot/compare/v26.12.3...v26.18.0
[v26.18.1]: https://github.com/tobymao/sqlglot/compare/v26.18.0...v26.18.1

8
CONTRIBUTING.md
View file

@ -8,9 +8,9 @@ easy and transparent as possible, whether it's:
- Submitting a fix
- Proposing new features
## We develop with Github
## We develop with GitHub
We use github to host code, to track issues and feature requests, as well as accept pull requests.
We use GitHub to host code, to track issues and feature requests, as well as accept pull requests.
## Finding tasks to work on
@ -39,7 +39,7 @@ to share any relevant context and increase its chances of getting merged.
Note: make sure to follow the [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) guidelines when creating a PR.
## Report bugs using Github's [issues](https://github.com/tobymao/sqlglot/issues)
## Report bugs using GitHub's [issues](https://github.com/tobymao/sqlglot/issues)
We use GitHub issues to track public bugs. Report a bug by opening a new issue.
@ -54,7 +54,7 @@ We use GitHub issues to track public bugs. Report a bug by opening a new issue.
- Notes (possibly including why you think this might be happening, or stuff you tried that didn't work)
- References (e.g. documentation pages related to the issue)
## Start a discussion using Github's [discussions](https://github.com/tobymao/sqlglot/discussions)
## Start a discussion using GitHub's [discussions](https://github.com/tobymao/sqlglot/discussions)
[We use GitHub discussions](https://github.com/tobymao/sqlglot/discussions/190) to discuss about the current state
of the code. If you want to propose a new feature, this is the right place to do it. Just start a discussion, and

4
README.md
View file

@ -1,6 +1,6 @@
![SQLGlot logo](sqlglot.png)
SQLGlot is a no-dependency SQL parser, transpiler, optimizer, and engine. It can be used to format SQL or translate between [24 different dialects](https://github.com/tobymao/sqlglot/blob/main/sqlglot/dialects/__init__.py) like [DuckDB](https://duckdb.org/), [Presto](https://prestodb.io/) / [Trino](https://trino.io/), [Spark](https://spark.apache.org/) / [Databricks](https://www.databricks.com/), [Snowflake](https://www.snowflake.com/en/), and [BigQuery](https://cloud.google.com/bigquery/). It aims to read a wide variety of SQL inputs and output syntactically and semantically correct SQL in the targeted dialects.
SQLGlot is a no-dependency SQL parser, transpiler, optimizer, and engine. It can be used to format SQL or translate between [27 different dialects](https://github.com/tobymao/sqlglot/blob/main/sqlglot/dialects/__init__.py) like [DuckDB](https://duckdb.org/), [Presto](https://prestodb.io/) / [Trino](https://trino.io/), [Spark](https://spark.apache.org/) / [Databricks](https://www.databricks.com/), [Snowflake](https://www.snowflake.com/en/), and [BigQuery](https://cloud.google.com/bigquery/). It aims to read a wide variety of SQL inputs and output syntactically and semantically correct SQL in the targeted dialects.
It is a very comprehensive generic SQL parser with a robust [test suite](https://github.com/tobymao/sqlglot/blob/main/tests/). It is also quite [performant](#benchmarks), while being written purely in Python.
@ -496,7 +496,7 @@ See also: [Writing a Python SQL engine from scratch](https://github.com/tobymao/
* [Apache Superset](https://github.com/apache/superset)
* [Dagster](https://github.com/dagster-io/dagster)
* [Fugue](https://github.com/fugue-project/fugue)
* [ibis](https://github.com/ibis-project/ibis)
* [Ibis](https://github.com/ibis-project/ibis)
* [mysql-mimic](https://github.com/kelsin/mysql-mimic)
* [Querybook](https://github.com/pinterest/querybook)
* [Quokka](https://github.com/marsupialtail/quokka)

2
docs/search.js
View file

File diff suppressed because one or more lines are too long

4
docs/sqlglot.html
View file

File diff suppressed because one or more lines are too long

12
docs/sqlglot/_version.html
View file

@ -84,8 +84,8 @@
</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a>
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;26.16.4&#39;</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">26</span><span class="p">,</span> <span class="mi">16</span><span class="p">,</span> <span class="mi">4</span><span class="p">)</span>
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;26.18.1&#39;</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">26</span><span class="p">,</span> <span class="mi">18</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
</span></pre></div>
@ -93,7 +93,7 @@
<section id="__version__">
<div class="attr variable">
<span class="name">__version__</span><span class="annotation">: str</span> =
<span class="default_value">&#39;26.16.4&#39;</span>
<span class="default_value">&#39;26.18.1&#39;</span>
</div>
@ -105,7 +105,7 @@
<section id="__version_tuple__">
<div class="attr variable">
<span class="name">__version_tuple__</span><span class="annotation">: object</span> =
<span class="default_value">(26, 16, 4)</span>
<span class="default_value">(26, 18, 1)</span>
</div>
@ -117,7 +117,7 @@
<section id="version">
<div class="attr variable">
<span class="name">version</span><span class="annotation">: str</span> =
<span class="default_value">&#39;26.16.4&#39;</span>
<span class="default_value">&#39;26.18.1&#39;</span>
</div>
@ -129,7 +129,7 @@
<section id="version_tuple">
<div class="attr variable">
<span class="name">version_tuple</span><span class="annotation">: object</span> =
<span class="default_value">(26, 16, 4)</span>
<span class="default_value">(26, 18, 1)</span>
</div>

58
docs/sqlglot/dialects.html
View file

@ -359,7 +359,7 @@ dialect implementations in order to understand how their various components can
<section id="Athena">
<div class="attr variable">
<span class="name">Athena</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930512520240&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834148751056&#39;&gt;</span>
</div>
@ -371,7 +371,7 @@ dialect implementations in order to understand how their various components can
<section id="BigQuery">
<div class="attr variable">
<span class="name">BigQuery</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930514521152&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834152605424&#39;&gt;</span>
</div>
@ -383,7 +383,7 @@ dialect implementations in order to understand how their various components can
<section id="ClickHouse">
<div class="attr variable">
<span class="name">ClickHouse</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930519272080&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834154348752&#39;&gt;</span>
</div>
@ -395,7 +395,7 @@ dialect implementations in order to understand how their various components can
<section id="Databricks">
<div class="attr variable">
<span class="name">Databricks</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930516854800&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834156445232&#39;&gt;</span>
</div>
@ -407,7 +407,7 @@ dialect implementations in order to understand how their various components can
<section id="Doris">
<div class="attr variable">
<span class="name">Doris</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930519102672&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834141819664&#39;&gt;</span>
</div>
@ -419,7 +419,7 @@ dialect implementations in order to understand how their various components can
<section id="Drill">
<div class="attr variable">
<span class="name">Drill</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930524552784&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834141823024&#39;&gt;</span>
</div>
@ -431,7 +431,7 @@ dialect implementations in order to understand how their various components can
<section id="Druid">
<div class="attr variable">
<span class="name">Druid</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930524553696&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834153558912&#39;&gt;</span>
</div>
@ -443,7 +443,7 @@ dialect implementations in order to understand how their various components can
<section id="DuckDB">
<div class="attr variable">
<span class="name">DuckDB</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930527083072&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834155577744&#39;&gt;</span>
</div>
@ -455,7 +455,7 @@ dialect implementations in order to understand how their various components can
<section id="Dune">
<div class="attr variable">
<span class="name">Dune</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930527085136&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834155579328&#39;&gt;</span>
</div>
@ -467,7 +467,7 @@ dialect implementations in order to understand how their various components can
<section id="Hive">
<div class="attr variable">
<span class="name">Hive</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930514780800&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834153801840&#39;&gt;</span>
</div>
@ -479,7 +479,7 @@ dialect implementations in order to understand how their various components can
<section id="Materialize">
<div class="attr variable">
<span class="name">Materialize</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930514770240&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834144070752&#39;&gt;</span>
</div>
@ -491,7 +491,7 @@ dialect implementations in order to understand how their various components can
<section id="MySQL">
<div class="attr variable">
<span class="name">MySQL</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930523412528&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834144075360&#39;&gt;</span>
</div>
@ -503,7 +503,7 @@ dialect implementations in order to understand how their various components can
<section id="Oracle">
<div class="attr variable">
<span class="name">Oracle</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930520980592&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834156307776&#39;&gt;</span>
</div>
@ -515,7 +515,7 @@ dialect implementations in order to understand how their various components can
<section id="Postgres">
<div class="attr variable">
<span class="name">Postgres</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930519622016&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834145972208&#39;&gt;</span>
</div>
@ -527,7 +527,7 @@ dialect implementations in order to understand how their various components can
<section id="Presto">
<div class="attr variable">
<span class="name">Presto</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930524419936&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834153153056&#39;&gt;</span>
</div>
@ -539,7 +539,7 @@ dialect implementations in order to understand how their various components can
<section id="PRQL">
<div class="attr variable">
<span class="name">PRQL</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930524423920&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834153154208&#39;&gt;</span>
</div>
@ -551,7 +551,7 @@ dialect implementations in order to understand how their various components can
<section id="Redshift">
<div class="attr variable">
<span class="name">Redshift</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930527795088&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834154538160&#39;&gt;</span>
</div>
@ -563,7 +563,7 @@ dialect implementations in order to understand how their various components can
<section id="RisingWave">
<div class="attr variable">
<span class="name">RisingWave</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930520236048&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834154668128&#39;&gt;</span>
</div>
@ -575,7 +575,7 @@ dialect implementations in order to understand how their various components can
<section id="Snowflake">
<div class="attr variable">
<span class="name">Snowflake</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930520224528&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834149397136&#39;&gt;</span>
</div>
@ -587,7 +587,7 @@ dialect implementations in order to understand how their various components can
<section id="Spark">
<div class="attr variable">
<span class="name">Spark</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930527493296&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834149389120&#39;&gt;</span>
</div>
@ -599,7 +599,7 @@ dialect implementations in order to understand how their various components can
<section id="Spark2">
<div class="attr variable">
<span class="name">Spark2</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930527460000&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834146082432&#39;&gt;</span>
</div>
@ -611,7 +611,7 @@ dialect implementations in order to understand how their various components can
<section id="SQLite">
<div class="attr variable">
<span class="name">SQLite</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930527452656&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834153479728&#39;&gt;</span>
</div>
@ -623,7 +623,7 @@ dialect implementations in order to understand how their various components can
<section id="StarRocks">
<div class="attr variable">
<span class="name">StarRocks</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930527588624&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834153476176&#39;&gt;</span>
</div>
@ -635,7 +635,7 @@ dialect implementations in order to understand how their various components can
<section id="Tableau">
<div class="attr variable">
<span class="name">Tableau</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930522889920&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834142598928&#39;&gt;</span>
</div>
@ -647,7 +647,7 @@ dialect implementations in order to understand how their various components can
<section id="Teradata">
<div class="attr variable">
<span class="name">Teradata</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930520249984&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834142598448&#39;&gt;</span>
</div>
@ -659,7 +659,7 @@ dialect implementations in order to understand how their various components can
<section id="Trino">
<div class="attr variable">
<span class="name">Trino</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930520246240&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834154814240&#39;&gt;</span>
</div>
@ -671,7 +671,7 @@ dialect implementations in order to understand how their various components can
<section id="TSQL">
<div class="attr variable">
<span class="name">TSQL</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930524096960&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834142228880&#39;&gt;</span>
</div>
@ -683,7 +683,7 @@ dialect implementations in order to understand how their various components can
<section id="Dialect">
<div class="attr variable">
<span class="name">Dialect</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930515373024&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834142236800&#39;&gt;</span>
</div>
@ -695,7 +695,7 @@ dialect implementations in order to understand how their various components can
<section id="Dialects">
<div class="attr variable">
<span class="name">Dialects</span> =
<span class="default_value">&lt;MagicMock id=&#39;139930515369760&#39;&gt;</span>
<span class="default_value">&lt;MagicMock id=&#39;139834142244720&#39;&gt;</span>
</div>

13
docs/sqlglot/dialects/athena.html
View file

File diff suppressed because one or more lines are too long

6306
docs/sqlglot/dialects/bigquery.html
View file

File diff suppressed because one or more lines are too long

7013
docs/sqlglot/dialects/clickhouse.html
View file

File diff suppressed because one or more lines are too long

15
docs/sqlglot/dialects/databricks.html
View file

File diff suppressed because one or more lines are too long

3010
docs/sqlglot/dialects/doris.html
View file

File diff suppressed because one or more lines are too long

15
docs/sqlglot/dialects/drill.html
View file

File diff suppressed because one or more lines are too long

39
docs/sqlglot/dialects/druid.html
View file

File diff suppressed because one or more lines are too long

5365
docs/sqlglot/dialects/duckdb.html
View file

File diff suppressed because one or more lines are too long

4
docs/sqlglot/dialects/dune.html
View file

File diff suppressed because one or more lines are too long

1714
docs/sqlglot/dialects/hive.html
View file

File diff suppressed because one or more lines are too long

9
docs/sqlglot/dialects/materialize.html
View file

File diff suppressed because one or more lines are too long

3124
docs/sqlglot/dialects/mysql.html
View file

File diff suppressed because one or more lines are too long

2026
docs/sqlglot/dialects/oracle.html
View file

File diff suppressed because one or more lines are too long

2700
docs/sqlglot/dialects/postgres.html
View file

File diff suppressed because one or more lines are too long

3040
docs/sqlglot/dialects/presto.html
View file

File diff suppressed because one or more lines are too long

9
docs/sqlglot/dialects/prql.html
View file

File diff suppressed because one or more lines are too long

1616
docs/sqlglot/dialects/redshift.html
View file

File diff suppressed because one or more lines are too long

15
docs/sqlglot/dialects/risingwave.html
View file

File diff suppressed because one or more lines are too long

2672
docs/sqlglot/dialects/snowflake.html
View file

File diff suppressed because one or more lines are too long

13
docs/sqlglot/dialects/spark.html
View file

File diff suppressed because one or more lines are too long

17
docs/sqlglot/dialects/spark2.html
View file

File diff suppressed because one or more lines are too long

1286
docs/sqlglot/dialects/sqlite.html
View file

File diff suppressed because one or more lines are too long

1222
docs/sqlglot/dialects/starrocks.html
View file

File diff suppressed because one or more lines are too long

13
docs/sqlglot/dialects/tableau.html
View file

File diff suppressed because one or more lines are too long

1211
docs/sqlglot/dialects/teradata.html
View file

File diff suppressed because one or more lines are too long

256
docs/sqlglot/dialects/trino.html
View file

File diff suppressed because one or more lines are too long

3291
docs/sqlglot/dialects/tsql.html
View file

File diff suppressed because one or more lines are too long

4
docs/sqlglot/executor/python.html
View file

File diff suppressed because one or more lines are too long

396
docs/sqlglot/executor/table.html
View file

@ -60,6 +60,9 @@
<li>
<a class="function" href="#Table.pop">pop</a>
</li>
<li>
<a class="function" href="#Table.to_pylist">to_pylist</a>
</li>
<li>
<a class="variable" href="#Table.width">width</a>
</li>
@ -177,119 +180,122 @@
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">pop</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a>
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a> <span class="nd">@property</span>
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">width</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a>
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">)</span>
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">to_pylist</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a> <span class="k">return</span> <span class="p">[</span><span class="nb">dict</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="n">row</span><span class="p">))</span> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">]</span>
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a>
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a> <span class="nd">@property</span>
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">width</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> <span class="k">return</span> <span class="n">TableIter</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">)</span>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">index</span><span class="p">):</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__repr__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">column</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="p">)</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">widths</span> <span class="o">=</span> <span class="p">{</span><span class="n">column</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">column</span><span class="p">)</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">}</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">lines</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot; &quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">)]</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">row</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="k">if</span> <span class="n">i</span> <span class="o">&gt;</span> <span class="mi">10</span><span class="p">:</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="k">break</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="n">lines</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="s2">&quot; &quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="nb">str</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="n">column</span><span class="p">])</span><span class="o">.</span><span class="n">rjust</span><span class="p">(</span><span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">])[</span><span class="mi">0</span> <span class="p">:</span> <span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="p">)</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="p">)</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="k">return</span> <span class="s2">&quot;</span><span class="se">\n</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">lines</span><span class="p">)</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a><span class="k">class</span><span class="w"> </span><span class="nc">TableIter</span><span class="p">:</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="k">return</span> <span class="n">TableIter</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">index</span><span class="p">):</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__repr__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">column</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="p">)</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">widths</span> <span class="o">=</span> <span class="p">{</span><span class="n">column</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">column</span><span class="p">)</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">}</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="n">lines</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot; &quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">)]</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">row</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">if</span> <span class="n">i</span> <span class="o">&gt;</span> <span class="mi">10</span><span class="p">:</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="k">break</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="n">lines</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="s2">&quot; &quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="nb">str</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="n">column</span><span class="p">])</span><span class="o">.</span><span class="n">rjust</span><span class="p">(</span><span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">])[</span><span class="mi">0</span> <span class="p">:</span> <span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="p">)</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="p">)</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="k">return</span> <span class="s2">&quot;</span><span class="se">\n</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">lines</span><span class="p">)</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a><span class="k">class</span><span class="w"> </span><span class="nc">TableIter</span><span class="p">:</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__next__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">+=</span> <span class="mi">1</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">&lt;</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">):</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">index</span><span class="p">]</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="k">raise</span> <span class="ne">StopIteration</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a><span class="k">class</span><span class="w"> </span><span class="nc">RangeReader</span><span class="p">:</span>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__next__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">+=</span> <span class="mi">1</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">&lt;</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">):</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">index</span><span class="p">]</span>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="k">raise</span> <span class="ne">StopIteration</span>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a><span class="k">class</span><span class="w"> </span><span class="nc">RangeReader</span><span class="p">:</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">return</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="n">column</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a><span class="k">class</span><span class="w"> </span><span class="nc">RowReader</span><span class="p">:</span>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">column</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">column_range</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="p">}</span>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="kc">None</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">row</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="k">return</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="n">column</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a><span class="k">class</span><span class="w"> </span><span class="nc">RowReader</span><span class="p">:</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">column</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">column_range</span>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="p">}</span>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="kc">None</span>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Tables</span><span class="p">(</span><span class="n">AbstractMappingSchema</span><span class="p">):</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="k">pass</span>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">row</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a><span class="k">def</span><span class="w"> </span><span class="nf">ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Tables</span><span class="p">:</span>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="k">return</span> <span class="n">Tables</span><span class="p">(</span><span class="n">_ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">))</span>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Tables</span><span class="p">(</span><span class="n">AbstractMappingSchema</span><span class="p">):</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="k">pass</span>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">d</span><span class="p">:</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="k">return</span> <span class="p">{}</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="n">depth</span> <span class="o">=</span> <span class="n">dict_depth</span><span class="p">(</span><span class="n">d</span><span class="p">)</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">if</span> <span class="n">depth</span> <span class="o">&gt;</span> <span class="mi">1</span><span class="p">:</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="k">return</span> <span class="p">{</span>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">normalize_name</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">is_table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="n">_ensure_tables</span><span class="p">(</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">v</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="p">)</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="p">}</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">result</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">for</span> <span class="n">table_name</span><span class="p">,</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">normalize_name</span><span class="p">(</span><span class="n">table_name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">name</span>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="n">Table</span><span class="p">):</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="n">result</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">table</span> <span class="o">=</span> <span class="p">[</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="p">{</span>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="n">normalize_name</span><span class="p">(</span><span class="n">column_name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="n">value</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">for</span> <span class="n">column_name</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">row</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="p">}</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">table</span>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="p">]</span>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="n">column_names</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">column_name</span> <span class="k">for</span> <span class="n">column_name</span> <span class="ow">in</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="k">if</span> <span class="n">table</span> <span class="k">else</span> <span class="p">()</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="n">rows</span> <span class="o">=</span> <span class="p">[</span><span class="nb">tuple</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">column_names</span><span class="p">)</span> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">table</span><span class="p">]</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="n">result</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="n">column_names</span><span class="p">,</span> <span class="n">rows</span><span class="o">=</span><span class="n">rows</span><span class="p">)</span>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">return</span> <span class="n">result</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a><span class="k">def</span><span class="w"> </span><span class="nf">ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Tables</span><span class="p">:</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">return</span> <span class="n">Tables</span><span class="p">(</span><span class="n">_ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">))</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">d</span><span class="p">:</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">return</span> <span class="p">{}</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">depth</span> <span class="o">=</span> <span class="n">dict_depth</span><span class="p">(</span><span class="n">d</span><span class="p">)</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">if</span> <span class="n">depth</span> <span class="o">&gt;</span> <span class="mi">1</span><span class="p">:</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="k">return</span> <span class="p">{</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="n">normalize_name</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">is_table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="n">_ensure_tables</span><span class="p">(</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">v</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="p">)</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="p">}</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="n">result</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">for</span> <span class="n">table_name</span><span class="p">,</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">normalize_name</span><span class="p">(</span><span class="n">table_name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">name</span>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="n">Table</span><span class="p">):</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="n">result</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="n">table</span> <span class="o">=</span> <span class="p">[</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="p">{</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">normalize_name</span><span class="p">(</span><span class="n">column_name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="n">value</span>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">for</span> <span class="n">column_name</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">row</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="p">}</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">table</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="p">]</span>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="n">column_names</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">column_name</span> <span class="k">for</span> <span class="n">column_name</span> <span class="ow">in</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="k">if</span> <span class="n">table</span> <span class="k">else</span> <span class="p">()</span>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">rows</span> <span class="o">=</span> <span class="p">[</span><span class="nb">tuple</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">column_names</span><span class="p">)</span> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">table</span><span class="p">]</span>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">result</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">Table</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="n">column_names</span><span class="p">,</span> <span class="n">rows</span><span class="o">=</span><span class="n">rows</span><span class="p">)</span>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="k">return</span> <span class="n">result</span>
</span></pre></div>
@ -330,39 +336,42 @@
</span><span id="Table-33"><a href="#Table-33"><span class="linenos">33</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">pop</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table-34"><a href="#Table-34"><span class="linenos">34</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
</span><span id="Table-35"><a href="#Table-35"><span class="linenos">35</span></a>
</span><span id="Table-36"><a href="#Table-36"><span class="linenos">36</span></a> <span class="nd">@property</span>
</span><span id="Table-37"><a href="#Table-37"><span class="linenos">37</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">width</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table-38"><a href="#Table-38"><span class="linenos">38</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
</span><span id="Table-39"><a href="#Table-39"><span class="linenos">39</span></a>
</span><span id="Table-40"><a href="#Table-40"><span class="linenos">40</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table-41"><a href="#Table-41"><span class="linenos">41</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">)</span>
</span><span id="Table-36"><a href="#Table-36"><span class="linenos">36</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">to_pylist</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table-37"><a href="#Table-37"><span class="linenos">37</span></a> <span class="k">return</span> <span class="p">[</span><span class="nb">dict</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="n">row</span><span class="p">))</span> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">]</span>
</span><span id="Table-38"><a href="#Table-38"><span class="linenos">38</span></a>
</span><span id="Table-39"><a href="#Table-39"><span class="linenos">39</span></a> <span class="nd">@property</span>
</span><span id="Table-40"><a href="#Table-40"><span class="linenos">40</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">width</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table-41"><a href="#Table-41"><span class="linenos">41</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
</span><span id="Table-42"><a href="#Table-42"><span class="linenos">42</span></a>
</span><span id="Table-43"><a href="#Table-43"><span class="linenos">43</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table-44"><a href="#Table-44"><span class="linenos">44</span></a> <span class="k">return</span> <span class="n">TableIter</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
</span><span id="Table-43"><a href="#Table-43"><span class="linenos">43</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table-44"><a href="#Table-44"><span class="linenos">44</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">)</span>
</span><span id="Table-45"><a href="#Table-45"><span class="linenos">45</span></a>
</span><span id="Table-46"><a href="#Table-46"><span class="linenos">46</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">index</span><span class="p">):</span>
</span><span id="Table-47"><a href="#Table-47"><span class="linenos">47</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
</span><span id="Table-48"><a href="#Table-48"><span class="linenos">48</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span>
</span><span id="Table-49"><a href="#Table-49"><span class="linenos">49</span></a>
</span><span id="Table-50"><a href="#Table-50"><span class="linenos">50</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__repr__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table-51"><a href="#Table-51"><span class="linenos">51</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span>
</span><span id="Table-52"><a href="#Table-52"><span class="linenos">52</span></a> <span class="n">column</span>
</span><span id="Table-53"><a href="#Table-53"><span class="linenos">53</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
</span><span id="Table-54"><a href="#Table-54"><span class="linenos">54</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span>
</span><span id="Table-55"><a href="#Table-55"><span class="linenos">55</span></a> <span class="p">)</span>
</span><span id="Table-56"><a href="#Table-56"><span class="linenos">56</span></a> <span class="n">widths</span> <span class="o">=</span> <span class="p">{</span><span class="n">column</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">column</span><span class="p">)</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">}</span>
</span><span id="Table-57"><a href="#Table-57"><span class="linenos">57</span></a> <span class="n">lines</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot; &quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">)]</span>
</span><span id="Table-58"><a href="#Table-58"><span class="linenos">58</span></a>
</span><span id="Table-59"><a href="#Table-59"><span class="linenos">59</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">row</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table-60"><a href="#Table-60"><span class="linenos">60</span></a> <span class="k">if</span> <span class="n">i</span> <span class="o">&gt;</span> <span class="mi">10</span><span class="p">:</span>
</span><span id="Table-61"><a href="#Table-61"><span class="linenos">61</span></a> <span class="k">break</span>
</span><span id="Table-62"><a href="#Table-62"><span class="linenos">62</span></a>
</span><span id="Table-63"><a href="#Table-63"><span class="linenos">63</span></a> <span class="n">lines</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
</span><span id="Table-64"><a href="#Table-64"><span class="linenos">64</span></a> <span class="s2">&quot; &quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span>
</span><span id="Table-65"><a href="#Table-65"><span class="linenos">65</span></a> <span class="nb">str</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="n">column</span><span class="p">])</span><span class="o">.</span><span class="n">rjust</span><span class="p">(</span><span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">])[</span><span class="mi">0</span> <span class="p">:</span> <span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span>
</span><span id="Table-66"><a href="#Table-66"><span class="linenos">66</span></a> <span class="p">)</span>
</span><span id="Table-67"><a href="#Table-67"><span class="linenos">67</span></a> <span class="p">)</span>
</span><span id="Table-68"><a href="#Table-68"><span class="linenos">68</span></a> <span class="k">return</span> <span class="s2">&quot;</span><span class="se">\n</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">lines</span><span class="p">)</span>
</span><span id="Table-46"><a href="#Table-46"><span class="linenos">46</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table-47"><a href="#Table-47"><span class="linenos">47</span></a> <span class="k">return</span> <span class="n">TableIter</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
</span><span id="Table-48"><a href="#Table-48"><span class="linenos">48</span></a>
</span><span id="Table-49"><a href="#Table-49"><span class="linenos">49</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">index</span><span class="p">):</span>
</span><span id="Table-50"><a href="#Table-50"><span class="linenos">50</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">[</span><span class="n">index</span><span class="p">]</span>
</span><span id="Table-51"><a href="#Table-51"><span class="linenos">51</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">reader</span>
</span><span id="Table-52"><a href="#Table-52"><span class="linenos">52</span></a>
</span><span id="Table-53"><a href="#Table-53"><span class="linenos">53</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__repr__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table-54"><a href="#Table-54"><span class="linenos">54</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span>
</span><span id="Table-55"><a href="#Table-55"><span class="linenos">55</span></a> <span class="n">column</span>
</span><span id="Table-56"><a href="#Table-56"><span class="linenos">56</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
</span><span id="Table-57"><a href="#Table-57"><span class="linenos">57</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">column_range</span>
</span><span id="Table-58"><a href="#Table-58"><span class="linenos">58</span></a> <span class="p">)</span>
</span><span id="Table-59"><a href="#Table-59"><span class="linenos">59</span></a> <span class="n">widths</span> <span class="o">=</span> <span class="p">{</span><span class="n">column</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">column</span><span class="p">)</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">}</span>
</span><span id="Table-60"><a href="#Table-60"><span class="linenos">60</span></a> <span class="n">lines</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot; &quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">)]</span>
</span><span id="Table-61"><a href="#Table-61"><span class="linenos">61</span></a>
</span><span id="Table-62"><a href="#Table-62"><span class="linenos">62</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">row</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table-63"><a href="#Table-63"><span class="linenos">63</span></a> <span class="k">if</span> <span class="n">i</span> <span class="o">&gt;</span> <span class="mi">10</span><span class="p">:</span>
</span><span id="Table-64"><a href="#Table-64"><span class="linenos">64</span></a> <span class="k">break</span>
</span><span id="Table-65"><a href="#Table-65"><span class="linenos">65</span></a>
</span><span id="Table-66"><a href="#Table-66"><span class="linenos">66</span></a> <span class="n">lines</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
</span><span id="Table-67"><a href="#Table-67"><span class="linenos">67</span></a> <span class="s2">&quot; &quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span>
</span><span id="Table-68"><a href="#Table-68"><span class="linenos">68</span></a> <span class="nb">str</span><span class="p">(</span><span class="n">row</span><span class="p">[</span><span class="n">column</span><span class="p">])</span><span class="o">.</span><span class="n">rjust</span><span class="p">(</span><span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">])[</span><span class="mi">0</span> <span class="p">:</span> <span class="n">widths</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span>
</span><span id="Table-69"><a href="#Table-69"><span class="linenos">69</span></a> <span class="p">)</span>
</span><span id="Table-70"><a href="#Table-70"><span class="linenos">70</span></a> <span class="p">)</span>
</span><span id="Table-71"><a href="#Table-71"><span class="linenos">71</span></a> <span class="k">return</span> <span class="s2">&quot;</span><span class="se">\n</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">lines</span><span class="p">)</span>
</span></pre></div>
@ -509,6 +518,25 @@
</div>
<div id="Table.to_pylist" class="classattr">
<input id="Table.to_pylist-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<div class="attr function">
<span class="def">def</span>
<span class="name">to_pylist</span><span class="signature pdoc-code condensed">(<span class="param"><span class="bp">self</span></span><span class="return-annotation">):</span></span>
<label class="view-source-button" for="Table.to_pylist-view-source"><span>View Source</span></label>
</div>
<a class="headerlink" href="#Table.to_pylist"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="Table.to_pylist-36"><a href="#Table.to_pylist-36"><span class="linenos">36</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">to_pylist</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table.to_pylist-37"><a href="#Table.to_pylist-37"><span class="linenos">37</span></a> <span class="k">return</span> <span class="p">[</span><span class="nb">dict</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">,</span> <span class="n">row</span><span class="p">))</span> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">rows</span><span class="p">]</span>
</span></pre></div>
</div>
<div id="Table.width" class="classattr">
<input id="Table.width-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
@ -519,9 +547,9 @@
</div>
<a class="headerlink" href="#Table.width"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="Table.width-36"><a href="#Table.width-36"><span class="linenos">36</span></a> <span class="nd">@property</span>
</span><span id="Table.width-37"><a href="#Table.width-37"><span class="linenos">37</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">width</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table.width-38"><a href="#Table.width-38"><span class="linenos">38</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="Table.width-39"><a href="#Table.width-39"><span class="linenos">39</span></a> <span class="nd">@property</span>
</span><span id="Table.width-40"><a href="#Table.width-40"><span class="linenos">40</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">width</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="Table.width-41"><a href="#Table.width-41"><span class="linenos">41</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
</span></pre></div>
@ -540,19 +568,19 @@
</div>
<a class="headerlink" href="#TableIter"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="TableIter-71"><a href="#TableIter-71"><span class="linenos">71</span></a><span class="k">class</span><span class="w"> </span><span class="nc">TableIter</span><span class="p">:</span>
</span><span id="TableIter-72"><a href="#TableIter-72"><span class="linenos">72</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
</span><span id="TableIter-73"><a href="#TableIter-73"><span class="linenos">73</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="TableIter-74"><a href="#TableIter-74"><span class="linenos">74</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span>
</span><span id="TableIter-75"><a href="#TableIter-75"><span class="linenos">75</span></a>
</span><span id="TableIter-76"><a href="#TableIter-76"><span class="linenos">76</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="TableIter-77"><a href="#TableIter-77"><span class="linenos">77</span></a> <span class="k">return</span> <span class="bp">self</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="TableIter-74"><a href="#TableIter-74"><span class="linenos">74</span></a><span class="k">class</span><span class="w"> </span><span class="nc">TableIter</span><span class="p">:</span>
</span><span id="TableIter-75"><a href="#TableIter-75"><span class="linenos">75</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
</span><span id="TableIter-76"><a href="#TableIter-76"><span class="linenos">76</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="TableIter-77"><a href="#TableIter-77"><span class="linenos">77</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span>
</span><span id="TableIter-78"><a href="#TableIter-78"><span class="linenos">78</span></a>
</span><span id="TableIter-79"><a href="#TableIter-79"><span class="linenos">79</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__next__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="TableIter-80"><a href="#TableIter-80"><span class="linenos">80</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">+=</span> <span class="mi">1</span>
</span><span id="TableIter-81"><a href="#TableIter-81"><span class="linenos">81</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">&lt;</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">):</span>
</span><span id="TableIter-82"><a href="#TableIter-82"><span class="linenos">82</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">index</span><span class="p">]</span>
</span><span id="TableIter-83"><a href="#TableIter-83"><span class="linenos">83</span></a> <span class="k">raise</span> <span class="ne">StopIteration</span>
</span><span id="TableIter-79"><a href="#TableIter-79"><span class="linenos">79</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="TableIter-80"><a href="#TableIter-80"><span class="linenos">80</span></a> <span class="k">return</span> <span class="bp">self</span>
</span><span id="TableIter-81"><a href="#TableIter-81"><span class="linenos">81</span></a>
</span><span id="TableIter-82"><a href="#TableIter-82"><span class="linenos">82</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__next__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="TableIter-83"><a href="#TableIter-83"><span class="linenos">83</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">+=</span> <span class="mi">1</span>
</span><span id="TableIter-84"><a href="#TableIter-84"><span class="linenos">84</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">&lt;</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">):</span>
</span><span id="TableIter-85"><a href="#TableIter-85"><span class="linenos">85</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">index</span><span class="p">]</span>
</span><span id="TableIter-86"><a href="#TableIter-86"><span class="linenos">86</span></a> <span class="k">raise</span> <span class="ne">StopIteration</span>
</span></pre></div>
@ -568,9 +596,9 @@
</div>
<a class="headerlink" href="#TableIter.__init__"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="TableIter.__init__-72"><a href="#TableIter.__init__-72"><span class="linenos">72</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
</span><span id="TableIter.__init__-73"><a href="#TableIter.__init__-73"><span class="linenos">73</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="TableIter.__init__-74"><a href="#TableIter.__init__-74"><span class="linenos">74</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="TableIter.__init__-75"><a href="#TableIter.__init__-75"><span class="linenos">75</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
</span><span id="TableIter.__init__-76"><a href="#TableIter.__init__-76"><span class="linenos">76</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="TableIter.__init__-77"><a href="#TableIter.__init__-77"><span class="linenos">77</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">index</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span>
</span></pre></div>
@ -611,16 +639,16 @@
</div>
<a class="headerlink" href="#RangeReader"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="RangeReader-86"><a href="#RangeReader-86"><span class="linenos">86</span></a><span class="k">class</span><span class="w"> </span><span class="nc">RangeReader</span><span class="p">:</span>
</span><span id="RangeReader-87"><a href="#RangeReader-87"><span class="linenos">87</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
</span><span id="RangeReader-88"><a href="#RangeReader-88"><span class="linenos">88</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="RangeReader-89"><a href="#RangeReader-89"><span class="linenos">89</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
</span><span id="RangeReader-90"><a href="#RangeReader-90"><span class="linenos">90</span></a>
</span><span id="RangeReader-91"><a href="#RangeReader-91"><span class="linenos">91</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="RangeReader-92"><a href="#RangeReader-92"><span class="linenos">92</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="RangeReader-89"><a href="#RangeReader-89"><span class="linenos">89</span></a><span class="k">class</span><span class="w"> </span><span class="nc">RangeReader</span><span class="p">:</span>
</span><span id="RangeReader-90"><a href="#RangeReader-90"><span class="linenos">90</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
</span><span id="RangeReader-91"><a href="#RangeReader-91"><span class="linenos">91</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="RangeReader-92"><a href="#RangeReader-92"><span class="linenos">92</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
</span><span id="RangeReader-93"><a href="#RangeReader-93"><span class="linenos">93</span></a>
</span><span id="RangeReader-94"><a href="#RangeReader-94"><span class="linenos">94</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
</span><span id="RangeReader-95"><a href="#RangeReader-95"><span class="linenos">95</span></a> <span class="k">return</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="n">column</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
</span><span id="RangeReader-94"><a href="#RangeReader-94"><span class="linenos">94</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="RangeReader-95"><a href="#RangeReader-95"><span class="linenos">95</span></a> <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
</span><span id="RangeReader-96"><a href="#RangeReader-96"><span class="linenos">96</span></a>
</span><span id="RangeReader-97"><a href="#RangeReader-97"><span class="linenos">97</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
</span><span id="RangeReader-98"><a href="#RangeReader-98"><span class="linenos">98</span></a> <span class="k">return</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">table</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="n">column</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">range</span><span class="p">)</span>
</span></pre></div>
@ -636,9 +664,9 @@
</div>
<a class="headerlink" href="#RangeReader.__init__"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="RangeReader.__init__-87"><a href="#RangeReader.__init__-87"><span class="linenos">87</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
</span><span id="RangeReader.__init__-88"><a href="#RangeReader.__init__-88"><span class="linenos">88</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="RangeReader.__init__-89"><a href="#RangeReader.__init__-89"><span class="linenos">89</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="RangeReader.__init__-90"><a href="#RangeReader.__init__-90"><span class="linenos">90</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">):</span>
</span><span id="RangeReader.__init__-91"><a href="#RangeReader.__init__-91"><span class="linenos">91</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">table</span> <span class="o">=</span> <span class="n">table</span>
</span><span id="RangeReader.__init__-92"><a href="#RangeReader.__init__-92"><span class="linenos">92</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">range</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
</span></pre></div>
@ -679,15 +707,15 @@
</div>
<a class="headerlink" href="#RowReader"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="RowReader-98"><a href="#RowReader-98"><span class="linenos"> 98</span></a><span class="k">class</span><span class="w"> </span><span class="nc">RowReader</span><span class="p">:</span>
</span><span id="RowReader-99"><a href="#RowReader-99"><span class="linenos"> 99</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="RowReader-100"><a href="#RowReader-100"><span class="linenos">100</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="RowReader-101"><a href="#RowReader-101"><span class="linenos">101</span></a> <span class="n">column</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">column_range</span>
</span><span id="RowReader-102"><a href="#RowReader-102"><span class="linenos">102</span></a> <span class="p">}</span>
</span><span id="RowReader-103"><a href="#RowReader-103"><span class="linenos">103</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="kc">None</span>
</span><span id="RowReader-104"><a href="#RowReader-104"><span class="linenos">104</span></a>
</span><span id="RowReader-105"><a href="#RowReader-105"><span class="linenos">105</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
</span><span id="RowReader-106"><a href="#RowReader-106"><span class="linenos">106</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">row</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="RowReader-101"><a href="#RowReader-101"><span class="linenos">101</span></a><span class="k">class</span><span class="w"> </span><span class="nc">RowReader</span><span class="p">:</span>
</span><span id="RowReader-102"><a href="#RowReader-102"><span class="linenos">102</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="RowReader-103"><a href="#RowReader-103"><span class="linenos">103</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="RowReader-104"><a href="#RowReader-104"><span class="linenos">104</span></a> <span class="n">column</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">column_range</span>
</span><span id="RowReader-105"><a href="#RowReader-105"><span class="linenos">105</span></a> <span class="p">}</span>
</span><span id="RowReader-106"><a href="#RowReader-106"><span class="linenos">106</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="kc">None</span>
</span><span id="RowReader-107"><a href="#RowReader-107"><span class="linenos">107</span></a>
</span><span id="RowReader-108"><a href="#RowReader-108"><span class="linenos">108</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column</span><span class="p">):</span>
</span><span id="RowReader-109"><a href="#RowReader-109"><span class="linenos">109</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">row</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">columns</span><span class="p">[</span><span class="n">column</span><span class="p">]]</span>
</span></pre></div>
@ -703,11 +731,11 @@
</div>
<a class="headerlink" href="#RowReader.__init__"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="RowReader.__init__-99"><a href="#RowReader.__init__-99"><span class="linenos"> 99</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="RowReader.__init__-100"><a href="#RowReader.__init__-100"><span class="linenos">100</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="RowReader.__init__-101"><a href="#RowReader.__init__-101"><span class="linenos">101</span></a> <span class="n">column</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">column_range</span>
</span><span id="RowReader.__init__-102"><a href="#RowReader.__init__-102"><span class="linenos">102</span></a> <span class="p">}</span>
</span><span id="RowReader.__init__-103"><a href="#RowReader.__init__-103"><span class="linenos">103</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="kc">None</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="RowReader.__init__-102"><a href="#RowReader.__init__-102"><span class="linenos">102</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">columns</span><span class="p">,</span> <span class="n">column_range</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="RowReader.__init__-103"><a href="#RowReader.__init__-103"><span class="linenos">103</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="RowReader.__init__-104"><a href="#RowReader.__init__-104"><span class="linenos">104</span></a> <span class="n">column</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">column</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">column_range</span> <span class="ow">or</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">column_range</span>
</span><span id="RowReader.__init__-105"><a href="#RowReader.__init__-105"><span class="linenos">105</span></a> <span class="p">}</span>
</span><span id="RowReader.__init__-106"><a href="#RowReader.__init__-106"><span class="linenos">106</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">row</span> <span class="o">=</span> <span class="kc">None</span>
</span></pre></div>
@ -748,8 +776,8 @@
</div>
<a class="headerlink" href="#Tables"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="Tables-109"><a href="#Tables-109"><span class="linenos">109</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Tables</span><span class="p">(</span><span class="n">AbstractMappingSchema</span><span class="p">):</span>
</span><span id="Tables-110"><a href="#Tables-110"><span class="linenos">110</span></a> <span class="k">pass</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="Tables-112"><a href="#Tables-112"><span class="linenos">112</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Tables</span><span class="p">(</span><span class="n">AbstractMappingSchema</span><span class="p">):</span>
</span><span id="Tables-113"><a href="#Tables-113"><span class="linenos">113</span></a> <span class="k">pass</span>
</span></pre></div>
@ -784,8 +812,8 @@
</div>
<a class="headerlink" href="#ensure_tables"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_tables-113"><a href="#ensure_tables-113"><span class="linenos">113</span></a><span class="k">def</span><span class="w"> </span><span class="nf">ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Tables</span><span class="p">:</span>
</span><span id="ensure_tables-114"><a href="#ensure_tables-114"><span class="linenos">114</span></a> <span class="k">return</span> <span class="n">Tables</span><span class="p">(</span><span class="n">_ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">))</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_tables-116"><a href="#ensure_tables-116"><span class="linenos">116</span></a><span class="k">def</span><span class="w"> </span><span class="nf">ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Tables</span><span class="p">:</span>
</span><span id="ensure_tables-117"><a href="#ensure_tables-117"><span class="linenos">117</span></a> <span class="k">return</span> <span class="n">Tables</span><span class="p">(</span><span class="n">_ensure_tables</span><span class="p">(</span><span class="n">d</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">))</span>
</span></pre></div>

23948
docs/sqlglot/expressions.html
View file

File diff suppressed because one or more lines are too long

25946
docs/sqlglot/generator.html
View file

File diff suppressed because one or more lines are too long

2
docs/sqlglot/helper.html
View file

@ -1956,7 +1956,7 @@ belong to some totally-ordered set.</p>
<section id="DATE_UNITS">
<div class="attr variable">
<span class="name">DATE_UNITS</span> =
<span class="default_value">{&#39;month&#39;, &#39;year&#39;, &#39;day&#39;, &#39;quarter&#39;, &#39;week&#39;, &#39;year_month&#39;}</span>
<span class="default_value">{&#39;year&#39;, &#39;year_month&#39;, &#39;week&#39;, &#39;day&#39;, &#39;month&#39;, &#39;quarter&#39;}</span>
</div>

2
docs/sqlglot/jsonpath.html
View file

@ -641,7 +641,7 @@
<div class="attr variable">
<span class="name">ALL_JSON_PATH_PARTS</span> =
<input id="ALL_JSON_PATH_PARTS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>&#39;&gt;}</span>
<label class="view-value-button pdoc-button" for="ALL_JSON_PATH_PARTS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="expressions.html#JSONPathSubscript">sqlglot.expressions.JSONPathSubscript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSelector">sqlglot.expressions.JSONPathSelector</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRoot">sqlglot.expressions.JSONPathRoot</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathSlice">sqlglot.expressions.JSONPathSlice</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathRecursive">sqlglot.expressions.JSONPathRecursive</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathScript">sqlglot.expressions.JSONPathScript</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathKey">sqlglot.expressions.JSONPathKey</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathFilter">sqlglot.expressions.JSONPathFilter</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathWildcard">sqlglot.expressions.JSONPathWildcard</a>&#39;&gt;, &lt;class &#39;<a href="expressions.html#JSONPathUnion">sqlglot.expressions.JSONPathUnion</a>&#39;&gt;}</span>
</div>

4
docs/sqlglot/optimizer/annotate_types.html
View file

File diff suppressed because one or more lines are too long

26
docs/sqlglot/optimizer/merge_subqueries.html
View file

@ -162,9 +162,9 @@
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">_rename_inner_sources</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="n">_merge_from</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">table</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">_merge_expressions</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">_merge_joins</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">_merge_where</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">_merge_joins</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">_merge_where</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">_merge_hints</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">_pop_cte</span><span class="p">(</span><span class="n">inner_scope</span><span class="p">)</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">clear_cache</span><span class="p">()</span>
@ -181,9 +181,9 @@
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="n">_rename_inner_sources</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="n">_merge_from</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">subquery</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="n">_merge_expressions</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="n">_merge_joins</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">_merge_where</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="n">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">_merge_joins</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">_merge_where</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">_merge_hints</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">clear_cache</span><span class="p">()</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a>
@ -581,7 +581,7 @@ queries if it would result in multiple table selects in a single query:</p>
<div class="attr variable">
<span class="name">UNMERGABLE_ARGS</span> =
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;match&#39;, &#39;windows&#39;, &#39;locks&#39;, &#39;limit&#39;, &#39;settings&#39;, &#39;laterals&#39;, &#39;pivots&#39;, &#39;group&#39;, &#39;into&#39;, &#39;distinct&#39;, &#39;distribute&#39;, &#39;offset&#39;, &#39;sort&#39;, &#39;with&#39;, &#39;connect&#39;, &#39;cluster&#39;, &#39;operation_modifiers&#39;, &#39;qualify&#39;, &#39;options&#39;, &#39;sample&#39;, &#39;format&#39;, &#39;prewhere&#39;, &#39;having&#39;, &#39;kind&#39;}</span>
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;limit&#39;, &#39;with&#39;, &#39;settings&#39;, &#39;into&#39;, &#39;group&#39;, &#39;distribute&#39;, &#39;prewhere&#39;, &#39;format&#39;, &#39;connect&#39;, &#39;laterals&#39;, &#39;match&#39;, &#39;qualify&#39;, &#39;windows&#39;, &#39;kind&#39;, &#39;pivots&#39;, &#39;having&#39;, &#39;sort&#39;, &#39;offset&#39;, &#39;sample&#39;, &#39;options&#39;, &#39;locks&#39;, &#39;distinct&#39;, &#39;cluster&#39;, &#39;operation_modifiers&#39;}</span>
</div>
@ -639,9 +639,9 @@ queries if it would result in multiple table selects in a single query:</p>
</span><span id="merge_ctes-93"><a href="#merge_ctes-93"><span class="linenos"> 93</span></a> <span class="n">_rename_inner_sources</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span>
</span><span id="merge_ctes-94"><a href="#merge_ctes-94"><span class="linenos"> 94</span></a> <span class="n">_merge_from</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">table</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span>
</span><span id="merge_ctes-95"><a href="#merge_ctes-95"><span class="linenos"> 95</span></a> <span class="n">_merge_expressions</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span>
</span><span id="merge_ctes-96"><a href="#merge_ctes-96"><span class="linenos"> 96</span></a> <span class="n">_merge_joins</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="merge_ctes-97"><a href="#merge_ctes-97"><span class="linenos"> 97</span></a> <span class="n">_merge_where</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="merge_ctes-98"><a href="#merge_ctes-98"><span class="linenos"> 98</span></a> <span class="n">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
</span><span id="merge_ctes-96"><a href="#merge_ctes-96"><span class="linenos"> 96</span></a> <span class="n">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
</span><span id="merge_ctes-97"><a href="#merge_ctes-97"><span class="linenos"> 97</span></a> <span class="n">_merge_joins</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="merge_ctes-98"><a href="#merge_ctes-98"><span class="linenos"> 98</span></a> <span class="n">_merge_where</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="merge_ctes-99"><a href="#merge_ctes-99"><span class="linenos"> 99</span></a> <span class="n">_merge_hints</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
</span><span id="merge_ctes-100"><a href="#merge_ctes-100"><span class="linenos">100</span></a> <span class="n">_pop_cte</span><span class="p">(</span><span class="n">inner_scope</span><span class="p">)</span>
</span><span id="merge_ctes-101"><a href="#merge_ctes-101"><span class="linenos">101</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">clear_cache</span><span class="p">()</span>
@ -673,9 +673,9 @@ queries if it would result in multiple table selects in a single query:</p>
</span><span id="merge_derived_tables-112"><a href="#merge_derived_tables-112"><span class="linenos">112</span></a> <span class="n">_rename_inner_sources</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span>
</span><span id="merge_derived_tables-113"><a href="#merge_derived_tables-113"><span class="linenos">113</span></a> <span class="n">_merge_from</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">subquery</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span>
</span><span id="merge_derived_tables-114"><a href="#merge_derived_tables-114"><span class="linenos">114</span></a> <span class="n">_merge_expressions</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">)</span>
</span><span id="merge_derived_tables-115"><a href="#merge_derived_tables-115"><span class="linenos">115</span></a> <span class="n">_merge_joins</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="merge_derived_tables-116"><a href="#merge_derived_tables-116"><span class="linenos">116</span></a> <span class="n">_merge_where</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="merge_derived_tables-117"><a href="#merge_derived_tables-117"><span class="linenos">117</span></a> <span class="n">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
</span><span id="merge_derived_tables-115"><a href="#merge_derived_tables-115"><span class="linenos">115</span></a> <span class="n">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
</span><span id="merge_derived_tables-116"><a href="#merge_derived_tables-116"><span class="linenos">116</span></a> <span class="n">_merge_joins</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="merge_derived_tables-117"><a href="#merge_derived_tables-117"><span class="linenos">117</span></a> <span class="n">_merge_where</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">)</span>
</span><span id="merge_derived_tables-118"><a href="#merge_derived_tables-118"><span class="linenos">118</span></a> <span class="n">_merge_hints</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
</span><span id="merge_derived_tables-119"><a href="#merge_derived_tables-119"><span class="linenos">119</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">clear_cache</span><span class="p">()</span>
</span><span id="merge_derived_tables-120"><a href="#merge_derived_tables-120"><span class="linenos">120</span></a>

807
docs/sqlglot/optimizer/pushdown_predicates.html
View file

@ -75,211 +75,224 @@
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.optimizer.normalize</span><span class="w"> </span><span class="kn">import</span> <span class="n">normalized</span>
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.optimizer.scope</span><span class="w"> </span><span class="kn">import</span> <span class="n">build_scope</span><span class="p">,</span> <span class="n">find_in_scope</span>
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.optimizer.simplify</span><span class="w"> </span><span class="kn">import</span> <span class="n">simplify</span>
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a>
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot</span><span class="w"> </span><span class="kn">import</span> <span class="n">Dialect</span>
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a>
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown_predicates</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="sd"> Rewrite sqlglot AST to pushdown predicates in FROMS and JOINS</span>
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a>
</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="sd"> Example:</span>
</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a><span class="sd"> &gt;&gt;&gt; sql = &quot;SELECT y.a AS a FROM (SELECT x.a AS a FROM x AS x) AS y WHERE y.a = 1&quot;</span>
</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(sql)</span>
</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a><span class="sd"> &gt;&gt;&gt; pushdown_predicates(expression).sql()</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="sd"> &#39;SELECT y.a AS a FROM (SELECT x.a AS a FROM x AS x WHERE x.a = 1) AS y WHERE TRUE&#39;</span>
</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a>
</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a><span class="sd"> Args:</span>
</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a><span class="sd"> expression (sqlglot.Expression): expression to optimize</span>
</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a><span class="sd"> Returns:</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a><span class="sd"> sqlglot.Expression: optimized expression</span>
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a> <span class="n">root</span> <span class="o">=</span> <span class="n">build_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a>
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a> <span class="k">if</span> <span class="n">root</span><span class="p">:</span>
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a> <span class="n">scope_ref_count</span> <span class="o">=</span> <span class="n">root</span><span class="o">.</span><span class="n">ref_count</span><span class="p">()</span>
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a>
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown_predicates</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="sd"> Rewrite sqlglot AST to pushdown predicates in FROMS and JOINS</span>
</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a>
</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a><span class="sd"> Example:</span>
</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a><span class="sd"> &gt;&gt;&gt; sql = &quot;SELECT y.a AS a FROM (SELECT x.a AS a FROM x AS x) AS y WHERE y.a = 1&quot;</span>
</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(sql)</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="sd"> &gt;&gt;&gt; pushdown_predicates(expression).sql()</span>
</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a><span class="sd"> &#39;SELECT y.a AS a FROM (SELECT x.a AS a FROM x AS x WHERE x.a = 1) AS y WHERE TRUE&#39;</span>
</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a>
</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a><span class="sd"> Args:</span>
</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a><span class="sd"> expression (sqlglot.Expression): expression to optimize</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a><span class="sd"> Returns:</span>
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a><span class="sd"> sqlglot.Expression: optimized expression</span>
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a> <span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.dialects.presto</span><span class="w"> </span><span class="kn">import</span> <span class="n">Presto</span>
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a>
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a> <span class="n">root</span> <span class="o">=</span> <span class="n">build_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a>
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="nb">reversed</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">root</span><span class="o">.</span><span class="n">traverse</span><span class="p">())):</span>
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a> <span class="n">select</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span>
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a> <span class="n">where</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;where&quot;</span><span class="p">)</span>
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a> <span class="k">if</span> <span class="n">where</span><span class="p">:</span>
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a> <span class="n">selected_sources</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span>
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a> <span class="n">join_index</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">join</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[])</span>
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a> <span class="p">}</span>
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a>
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a> <span class="c1"># a right join can only push down to itself and not the source FROM table</span>
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">source</span><span class="p">)</span> <span class="ow">in</span> <span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">)</span> <span class="ow">and</span> <span class="n">parent</span><span class="o">.</span><span class="n">side</span> <span class="o">==</span> <span class="s2">&quot;RIGHT&quot;</span><span class="p">:</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="n">selected_sources</span> <span class="o">=</span> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">source</span><span class="p">)}</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="k">break</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="n">pushdown</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">selected_sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">dialect</span><span class="p">,</span> <span class="n">join_index</span><span class="p">)</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="c1"># joins should only pushdown into itself, not to other joins</span>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="c1"># so we limit the selected sources to only itself</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]:</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">:</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">pushdown</span><span class="p">(</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;on&quot;</span><span class="p">),</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">name</span><span class="p">]},</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">scope_ref_count</span><span class="p">,</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">dialect</span><span class="p">,</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="p">)</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">dialect</span><span class="p">,</span> <span class="n">join_index</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">condition</span><span class="p">:</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="k">return</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">condition</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">simplify</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">))</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="n">cnf_like</span> <span class="o">=</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">)</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="n">predicates</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="n">condition</span><span class="o">.</span><span class="n">flatten</span><span class="p">()</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">if</span> <span class="n">cnf_like</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="k">else</span> <span class="p">[</span><span class="n">condition</span><span class="p">]</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="p">)</span>
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a> <span class="n">unnest_requires_cross_join</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">dialect</span><span class="p">,</span> <span class="n">Presto</span><span class="p">)</span>
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a>
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a> <span class="k">if</span> <span class="n">root</span><span class="p">:</span>
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a> <span class="n">scope_ref_count</span> <span class="o">=</span> <span class="n">root</span><span class="o">.</span><span class="n">ref_count</span><span class="p">()</span>
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a>
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="nb">reversed</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">root</span><span class="o">.</span><span class="n">traverse</span><span class="p">())):</span>
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a> <span class="n">select</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span>
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a> <span class="n">where</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;where&quot;</span><span class="p">)</span>
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a> <span class="k">if</span> <span class="n">where</span><span class="p">:</span>
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a> <span class="n">selected_sources</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span>
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="n">join_index</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">join</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[])</span>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="p">}</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> <span class="c1"># a right join can only push down to itself and not the source FROM table</span>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="c1"># presto, trino and athena don&#39;t support inner joins where the RHS is an UNNEST expression</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="n">pushdown_allowed</span> <span class="o">=</span> <span class="kc">True</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">source</span><span class="p">)</span> <span class="ow">in</span> <span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="k">if</span> <span class="n">parent</span><span class="o">.</span><span class="n">side</span> <span class="o">==</span> <span class="s2">&quot;RIGHT&quot;</span><span class="p">:</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="n">selected_sources</span> <span class="o">=</span> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">source</span><span class="p">)}</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="k">break</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Unnest</span><span class="p">)</span> <span class="ow">and</span> <span class="n">unnest_requires_cross_join</span><span class="p">:</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="n">pushdown_allowed</span> <span class="o">=</span> <span class="kc">False</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="k">break</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="k">if</span> <span class="n">pushdown_allowed</span><span class="p">:</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="n">pushdown</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">selected_sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">dialect</span><span class="p">,</span> <span class="n">join_index</span><span class="p">)</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="c1"># joins should only pushdown into itself, not to other joins</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="c1"># so we limit the selected sources to only itself</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]:</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">:</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="n">pushdown</span><span class="p">(</span>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;on&quot;</span><span class="p">),</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">name</span><span class="p">]},</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="n">scope_ref_count</span><span class="p">,</span>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="n">dialect</span><span class="p">,</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="p">)</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="k">if</span> <span class="n">cnf_like</span><span class="p">:</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="n">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">join_index</span><span class="o">=</span><span class="n">join_index</span><span class="p">)</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">join_index</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a><span class="sd"> If the predicates are in CNF like form, we can simply replace each block in the parent.</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="n">join_index</span> <span class="o">=</span> <span class="n">join_index</span> <span class="ow">or</span> <span class="p">{}</span>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">alias_or_name</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">predicate_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="c1"># Don&#39;t push the predicate if it references tables that appear in later joins</span>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">this_index</span> <span class="o">=</span> <span class="n">join_index</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">if</span> <span class="nb">all</span><span class="p">(</span><span class="n">join_index</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="o">&lt;</span> <span class="n">this_index</span> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">predicate_tables</span><span class="p">):</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="k">break</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">inner_predicate</span> <span class="o">=</span> <span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">)</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">if</span> <span class="n">find_in_scope</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">):</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">node</span><span class="o">.</span><span class="n">having</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a><span class="sd"> If the predicates are in DNF form, we can only push down conditions that are in all blocks.</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a><span class="sd"> Additionally, we can&#39;t remove predicates from their original form.</span>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="c1"># find all the tables that can be pushdown too</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="c1"># these are tables that are referenced in all blocks of a DNF</span>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="c1"># (a.x AND b.x) OR (a.y AND c.y)</span>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="c1"># only table a can be push down</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">pushdown_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="n">a_tables</span> <span class="o">&amp;=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">pushdown_tables</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">a_tables</span><span class="p">)</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">conditions</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="c1"># pushdown all predicates to their respective nodes</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">pushdown_tables</span><span class="p">):</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">nodes</span><span class="p">:</span>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">continue</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span><span class="p">(</span><span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">],</span> <span class="n">predicate</span><span class="p">)</span> <span class="k">if</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">conditions</span> <span class="k">else</span> <span class="n">predicate</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="p">)</span>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">conditions</span><span class="p">:</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="k">continue</span>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="n">predicate</span> <span class="o">=</span> <span class="n">conditions</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">dialect</span><span class="p">,</span> <span class="n">join_index</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">condition</span><span class="p">:</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="k">return</span>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">condition</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">simplify</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">))</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="n">cnf_like</span> <span class="o">=</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">)</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">predicates</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="n">condition</span><span class="o">.</span><span class="n">flatten</span><span class="p">()</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">if</span> <span class="n">cnf_like</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="k">else</span> <span class="p">[</span><span class="n">condition</span><span class="p">]</span>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="p">)</span>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="k">if</span> <span class="n">cnf_like</span><span class="p">:</span>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="n">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">join_index</span><span class="o">=</span><span class="n">join_index</span><span class="p">)</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="n">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">join_index</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a><span class="sd"> If the predicates are in CNF like form, we can simply replace each block in the parent.</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">join_index</span> <span class="o">=</span> <span class="n">join_index</span> <span class="ow">or</span> <span class="p">{}</span>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">alias_or_name</span>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="n">predicate_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="c1"># Don&#39;t push the predicate if it references tables that appear in later joins</span>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="n">this_index</span> <span class="o">=</span> <span class="n">join_index</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="k">if</span> <span class="nb">all</span><span class="p">(</span><span class="n">join_index</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="o">&lt;</span> <span class="n">this_index</span> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">predicate_tables</span><span class="p">):</span>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="k">break</span>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="n">inner_predicate</span> <span class="o">=</span> <span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">)</span>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="k">if</span> <span class="n">find_in_scope</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">):</span>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="n">node</span><span class="o">.</span><span class="n">having</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a><span class="sd"> If the predicates are in DNF form, we can only push down conditions that are in all blocks.</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a><span class="sd"> Additionally, we can&#39;t remove predicates from their original form.</span>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="c1"># find all the tables that can be pushdown too</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="c1"># these are tables that are referenced in all blocks of a DNF</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="c1"># (a.x AND b.x) OR (a.y AND c.y)</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="c1"># only table a can be push down</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">pushdown_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="n">a_tables</span> <span class="o">&amp;=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">pushdown_tables</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">a_tables</span><span class="p">)</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="n">conditions</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="c1"># pushdown all predicates to their respective nodes</span>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">pushdown_tables</span><span class="p">):</span>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">inner_predicate</span> <span class="o">=</span> <span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">)</span>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="k">if</span> <span class="n">find_in_scope</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">):</span>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">node</span><span class="o">.</span><span class="n">having</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a>
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a>
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a><span class="k">def</span><span class="w"> </span><span class="nf">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="n">where_condition</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">)</span>
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a>
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">tables</span><span class="p">):</span>
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">source</span> <span class="o">=</span> <span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="ow">or</span> <span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a>
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="c1"># if the predicate is in a where statement we can try to push it down</span>
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="c1"># we want to find the root join or from statement</span>
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">where_condition</span><span class="p">:</span>
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">nodes</span><span class="p">:</span>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="k">continue</span>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span><span class="p">(</span><span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">],</span> <span class="n">predicate</span><span class="p">)</span> <span class="k">if</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">conditions</span> <span class="k">else</span> <span class="n">predicate</span>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="p">)</span>
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a>
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">conditions</span><span class="p">:</span>
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="k">continue</span>
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a>
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="n">predicate</span> <span class="o">=</span> <span class="n">conditions</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a>
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="n">inner_predicate</span> <span class="o">=</span> <span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">)</span>
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">if</span> <span class="n">find_in_scope</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">):</span>
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="n">node</span><span class="o">.</span><span class="n">having</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a>
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a>
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="c1"># a node can reference a CTE which should be pushed down</span>
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">)</span>
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="k">if</span> <span class="n">with_</span> <span class="ow">and</span> <span class="n">with_</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="k">return</span> <span class="p">{}</span>
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a>
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="k">if</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="o">!=</span> <span class="s2">&quot;RIGHT&quot;</span><span class="p">:</span>
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="k">return</span> <span class="p">{}</span>
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="c1"># We can&#39;t push down window expressions</span>
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="n">has_window_expression</span> <span class="o">=</span> <span class="nb">any</span><span class="p">(</span>
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="p">)</span>
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="c1"># we can&#39;t push down predicates to select statements if they are referenced in</span>
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="c1"># multiple places.</span>
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;group&quot;</span><span class="p">)</span>
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="ow">and</span> <span class="n">scope_ref_count</span><span class="p">[</span><span class="nb">id</span><span class="p">(</span><span class="n">source</span><span class="p">)]</span> <span class="o">&lt;</span> <span class="mi">2</span>
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">has_window_expression</span>
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="p">):</span>
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="k">return</span> <span class="n">nodes</span>
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a>
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a>
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a><span class="k">def</span><span class="w"> </span><span class="nf">replace_aliases</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">predicate</span><span class="p">):</span>
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="n">aliases</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a>
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span>
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a>
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">_replace_alias</span><span class="p">(</span><span class="n">column</span><span class="p">):</span>
</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="ow">and</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">aliases</span><span class="p">:</span>
</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a> <span class="k">return</span> <span class="n">aliases</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a> <span class="k">return</span> <span class="n">column</span>
</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a>
</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a> <span class="k">return</span> <span class="n">predicate</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_replace_alias</span><span class="p">)</span>
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a><span class="k">def</span><span class="w"> </span><span class="nf">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="n">where_condition</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">)</span>
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a>
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">tables</span><span class="p">):</span>
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">source</span> <span class="o">=</span> <span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="ow">or</span> <span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a>
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="c1"># if the predicate is in a where statement we can try to push it down</span>
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="c1"># we want to find the root join or from statement</span>
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">where_condition</span><span class="p">:</span>
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a>
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="c1"># a node can reference a CTE which should be pushed down</span>
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">)</span>
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="k">if</span> <span class="n">with_</span> <span class="ow">and</span> <span class="n">with_</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="k">return</span> <span class="p">{}</span>
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a>
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="k">if</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="o">!=</span> <span class="s2">&quot;RIGHT&quot;</span><span class="p">:</span>
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="k">return</span> <span class="p">{}</span>
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="c1"># We can&#39;t push down window expressions</span>
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="n">has_window_expression</span> <span class="o">=</span> <span class="nb">any</span><span class="p">(</span>
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="p">)</span>
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="c1"># we can&#39;t push down predicates to select statements if they are referenced in</span>
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="c1"># multiple places.</span>
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;group&quot;</span><span class="p">)</span>
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="ow">and</span> <span class="n">scope_ref_count</span><span class="p">[</span><span class="nb">id</span><span class="p">(</span><span class="n">source</span><span class="p">)]</span> <span class="o">&lt;</span> <span class="mi">2</span>
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">has_window_expression</span>
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a> <span class="p">):</span>
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a> <span class="k">return</span> <span class="n">nodes</span>
</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a>
</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a>
</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a><span class="k">def</span><span class="w"> </span><span class="nf">replace_aliases</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">predicate</span><span class="p">):</span>
</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a> <span class="n">aliases</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a>
</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span>
</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a>
</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">_replace_alias</span><span class="p">(</span><span class="n">column</span><span class="p">):</span>
</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="ow">and</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">aliases</span><span class="p">:</span>
</span><span id="L-219"><a href="#L-219"><span class="linenos">219</span></a> <span class="k">return</span> <span class="n">aliases</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
</span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a> <span class="k">return</span> <span class="n">column</span>
</span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a>
</span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a> <span class="k">return</span> <span class="n">predicate</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_replace_alias</span><span class="p">)</span>
</span></pre></div>
@ -295,58 +308,70 @@
</div>
<a class="headerlink" href="#pushdown_predicates"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown_predicates-8"><a href="#pushdown_predicates-8"><span class="linenos"> 8</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown_predicates</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="pushdown_predicates-9"><a href="#pushdown_predicates-9"><span class="linenos"> 9</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="pushdown_predicates-10"><a href="#pushdown_predicates-10"><span class="linenos">10</span></a><span class="sd"> Rewrite sqlglot AST to pushdown predicates in FROMS and JOINS</span>
</span><span id="pushdown_predicates-11"><a href="#pushdown_predicates-11"><span class="linenos">11</span></a>
</span><span id="pushdown_predicates-12"><a href="#pushdown_predicates-12"><span class="linenos">12</span></a><span class="sd"> Example:</span>
</span><span id="pushdown_predicates-13"><a href="#pushdown_predicates-13"><span class="linenos">13</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="pushdown_predicates-14"><a href="#pushdown_predicates-14"><span class="linenos">14</span></a><span class="sd"> &gt;&gt;&gt; sql = &quot;SELECT y.a AS a FROM (SELECT x.a AS a FROM x AS x) AS y WHERE y.a = 1&quot;</span>
</span><span id="pushdown_predicates-15"><a href="#pushdown_predicates-15"><span class="linenos">15</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(sql)</span>
</span><span id="pushdown_predicates-16"><a href="#pushdown_predicates-16"><span class="linenos">16</span></a><span class="sd"> &gt;&gt;&gt; pushdown_predicates(expression).sql()</span>
</span><span id="pushdown_predicates-17"><a href="#pushdown_predicates-17"><span class="linenos">17</span></a><span class="sd"> &#39;SELECT y.a AS a FROM (SELECT x.a AS a FROM x AS x WHERE x.a = 1) AS y WHERE TRUE&#39;</span>
</span><span id="pushdown_predicates-18"><a href="#pushdown_predicates-18"><span class="linenos">18</span></a>
</span><span id="pushdown_predicates-19"><a href="#pushdown_predicates-19"><span class="linenos">19</span></a><span class="sd"> Args:</span>
</span><span id="pushdown_predicates-20"><a href="#pushdown_predicates-20"><span class="linenos">20</span></a><span class="sd"> expression (sqlglot.Expression): expression to optimize</span>
</span><span id="pushdown_predicates-21"><a href="#pushdown_predicates-21"><span class="linenos">21</span></a><span class="sd"> Returns:</span>
</span><span id="pushdown_predicates-22"><a href="#pushdown_predicates-22"><span class="linenos">22</span></a><span class="sd"> sqlglot.Expression: optimized expression</span>
</span><span id="pushdown_predicates-23"><a href="#pushdown_predicates-23"><span class="linenos">23</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="pushdown_predicates-24"><a href="#pushdown_predicates-24"><span class="linenos">24</span></a> <span class="n">root</span> <span class="o">=</span> <span class="n">build_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="pushdown_predicates-25"><a href="#pushdown_predicates-25"><span class="linenos">25</span></a>
</span><span id="pushdown_predicates-26"><a href="#pushdown_predicates-26"><span class="linenos">26</span></a> <span class="k">if</span> <span class="n">root</span><span class="p">:</span>
</span><span id="pushdown_predicates-27"><a href="#pushdown_predicates-27"><span class="linenos">27</span></a> <span class="n">scope_ref_count</span> <span class="o">=</span> <span class="n">root</span><span class="o">.</span><span class="n">ref_count</span><span class="p">()</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown_predicates-9"><a href="#pushdown_predicates-9"><span class="linenos"> 9</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown_predicates</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="pushdown_predicates-10"><a href="#pushdown_predicates-10"><span class="linenos">10</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="pushdown_predicates-11"><a href="#pushdown_predicates-11"><span class="linenos">11</span></a><span class="sd"> Rewrite sqlglot AST to pushdown predicates in FROMS and JOINS</span>
</span><span id="pushdown_predicates-12"><a href="#pushdown_predicates-12"><span class="linenos">12</span></a>
</span><span id="pushdown_predicates-13"><a href="#pushdown_predicates-13"><span class="linenos">13</span></a><span class="sd"> Example:</span>
</span><span id="pushdown_predicates-14"><a href="#pushdown_predicates-14"><span class="linenos">14</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="pushdown_predicates-15"><a href="#pushdown_predicates-15"><span class="linenos">15</span></a><span class="sd"> &gt;&gt;&gt; sql = &quot;SELECT y.a AS a FROM (SELECT x.a AS a FROM x AS x) AS y WHERE y.a = 1&quot;</span>
</span><span id="pushdown_predicates-16"><a href="#pushdown_predicates-16"><span class="linenos">16</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(sql)</span>
</span><span id="pushdown_predicates-17"><a href="#pushdown_predicates-17"><span class="linenos">17</span></a><span class="sd"> &gt;&gt;&gt; pushdown_predicates(expression).sql()</span>
</span><span id="pushdown_predicates-18"><a href="#pushdown_predicates-18"><span class="linenos">18</span></a><span class="sd"> &#39;SELECT y.a AS a FROM (SELECT x.a AS a FROM x AS x WHERE x.a = 1) AS y WHERE TRUE&#39;</span>
</span><span id="pushdown_predicates-19"><a href="#pushdown_predicates-19"><span class="linenos">19</span></a>
</span><span id="pushdown_predicates-20"><a href="#pushdown_predicates-20"><span class="linenos">20</span></a><span class="sd"> Args:</span>
</span><span id="pushdown_predicates-21"><a href="#pushdown_predicates-21"><span class="linenos">21</span></a><span class="sd"> expression (sqlglot.Expression): expression to optimize</span>
</span><span id="pushdown_predicates-22"><a href="#pushdown_predicates-22"><span class="linenos">22</span></a><span class="sd"> Returns:</span>
</span><span id="pushdown_predicates-23"><a href="#pushdown_predicates-23"><span class="linenos">23</span></a><span class="sd"> sqlglot.Expression: optimized expression</span>
</span><span id="pushdown_predicates-24"><a href="#pushdown_predicates-24"><span class="linenos">24</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="pushdown_predicates-25"><a href="#pushdown_predicates-25"><span class="linenos">25</span></a> <span class="kn">from</span><span class="w"> </span><span class="nn">sqlglot.dialects.presto</span><span class="w"> </span><span class="kn">import</span> <span class="n">Presto</span>
</span><span id="pushdown_predicates-26"><a href="#pushdown_predicates-26"><span class="linenos">26</span></a>
</span><span id="pushdown_predicates-27"><a href="#pushdown_predicates-27"><span class="linenos">27</span></a> <span class="n">root</span> <span class="o">=</span> <span class="n">build_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="pushdown_predicates-28"><a href="#pushdown_predicates-28"><span class="linenos">28</span></a>
</span><span id="pushdown_predicates-29"><a href="#pushdown_predicates-29"><span class="linenos">29</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="nb">reversed</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">root</span><span class="o">.</span><span class="n">traverse</span><span class="p">())):</span>
</span><span id="pushdown_predicates-30"><a href="#pushdown_predicates-30"><span class="linenos">30</span></a> <span class="n">select</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span>
</span><span id="pushdown_predicates-31"><a href="#pushdown_predicates-31"><span class="linenos">31</span></a> <span class="n">where</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;where&quot;</span><span class="p">)</span>
</span><span id="pushdown_predicates-32"><a href="#pushdown_predicates-32"><span class="linenos">32</span></a> <span class="k">if</span> <span class="n">where</span><span class="p">:</span>
</span><span id="pushdown_predicates-33"><a href="#pushdown_predicates-33"><span class="linenos">33</span></a> <span class="n">selected_sources</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span>
</span><span id="pushdown_predicates-34"><a href="#pushdown_predicates-34"><span class="linenos">34</span></a> <span class="n">join_index</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="pushdown_predicates-35"><a href="#pushdown_predicates-35"><span class="linenos">35</span></a> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">join</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[])</span>
</span><span id="pushdown_predicates-36"><a href="#pushdown_predicates-36"><span class="linenos">36</span></a> <span class="p">}</span>
</span><span id="pushdown_predicates-37"><a href="#pushdown_predicates-37"><span class="linenos">37</span></a>
</span><span id="pushdown_predicates-38"><a href="#pushdown_predicates-38"><span class="linenos">38</span></a> <span class="c1"># a right join can only push down to itself and not the source FROM table</span>
</span><span id="pushdown_predicates-39"><a href="#pushdown_predicates-39"><span class="linenos">39</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">source</span><span class="p">)</span> <span class="ow">in</span> <span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="pushdown_predicates-40"><a href="#pushdown_predicates-40"><span class="linenos">40</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
</span><span id="pushdown_predicates-41"><a href="#pushdown_predicates-41"><span class="linenos">41</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">)</span> <span class="ow">and</span> <span class="n">parent</span><span class="o">.</span><span class="n">side</span> <span class="o">==</span> <span class="s2">&quot;RIGHT&quot;</span><span class="p">:</span>
</span><span id="pushdown_predicates-42"><a href="#pushdown_predicates-42"><span class="linenos">42</span></a> <span class="n">selected_sources</span> <span class="o">=</span> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">source</span><span class="p">)}</span>
</span><span id="pushdown_predicates-43"><a href="#pushdown_predicates-43"><span class="linenos">43</span></a> <span class="k">break</span>
</span><span id="pushdown_predicates-44"><a href="#pushdown_predicates-44"><span class="linenos">44</span></a>
</span><span id="pushdown_predicates-45"><a href="#pushdown_predicates-45"><span class="linenos">45</span></a> <span class="n">pushdown</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">selected_sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">dialect</span><span class="p">,</span> <span class="n">join_index</span><span class="p">)</span>
</span><span id="pushdown_predicates-46"><a href="#pushdown_predicates-46"><span class="linenos">46</span></a>
</span><span id="pushdown_predicates-47"><a href="#pushdown_predicates-47"><span class="linenos">47</span></a> <span class="c1"># joins should only pushdown into itself, not to other joins</span>
</span><span id="pushdown_predicates-48"><a href="#pushdown_predicates-48"><span class="linenos">48</span></a> <span class="c1"># so we limit the selected sources to only itself</span>
</span><span id="pushdown_predicates-49"><a href="#pushdown_predicates-49"><span class="linenos">49</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]:</span>
</span><span id="pushdown_predicates-50"><a href="#pushdown_predicates-50"><span class="linenos">50</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
</span><span id="pushdown_predicates-51"><a href="#pushdown_predicates-51"><span class="linenos">51</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">:</span>
</span><span id="pushdown_predicates-52"><a href="#pushdown_predicates-52"><span class="linenos">52</span></a> <span class="n">pushdown</span><span class="p">(</span>
</span><span id="pushdown_predicates-53"><a href="#pushdown_predicates-53"><span class="linenos">53</span></a> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;on&quot;</span><span class="p">),</span>
</span><span id="pushdown_predicates-54"><a href="#pushdown_predicates-54"><span class="linenos">54</span></a> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">name</span><span class="p">]},</span>
</span><span id="pushdown_predicates-55"><a href="#pushdown_predicates-55"><span class="linenos">55</span></a> <span class="n">scope_ref_count</span><span class="p">,</span>
</span><span id="pushdown_predicates-56"><a href="#pushdown_predicates-56"><span class="linenos">56</span></a> <span class="n">dialect</span><span class="p">,</span>
</span><span id="pushdown_predicates-57"><a href="#pushdown_predicates-57"><span class="linenos">57</span></a> <span class="p">)</span>
</span><span id="pushdown_predicates-58"><a href="#pushdown_predicates-58"><span class="linenos">58</span></a>
</span><span id="pushdown_predicates-59"><a href="#pushdown_predicates-59"><span class="linenos">59</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="pushdown_predicates-29"><a href="#pushdown_predicates-29"><span class="linenos">29</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="pushdown_predicates-30"><a href="#pushdown_predicates-30"><span class="linenos">30</span></a> <span class="n">unnest_requires_cross_join</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">dialect</span><span class="p">,</span> <span class="n">Presto</span><span class="p">)</span>
</span><span id="pushdown_predicates-31"><a href="#pushdown_predicates-31"><span class="linenos">31</span></a>
</span><span id="pushdown_predicates-32"><a href="#pushdown_predicates-32"><span class="linenos">32</span></a> <span class="k">if</span> <span class="n">root</span><span class="p">:</span>
</span><span id="pushdown_predicates-33"><a href="#pushdown_predicates-33"><span class="linenos">33</span></a> <span class="n">scope_ref_count</span> <span class="o">=</span> <span class="n">root</span><span class="o">.</span><span class="n">ref_count</span><span class="p">()</span>
</span><span id="pushdown_predicates-34"><a href="#pushdown_predicates-34"><span class="linenos">34</span></a>
</span><span id="pushdown_predicates-35"><a href="#pushdown_predicates-35"><span class="linenos">35</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="nb">reversed</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">root</span><span class="o">.</span><span class="n">traverse</span><span class="p">())):</span>
</span><span id="pushdown_predicates-36"><a href="#pushdown_predicates-36"><span class="linenos">36</span></a> <span class="n">select</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span>
</span><span id="pushdown_predicates-37"><a href="#pushdown_predicates-37"><span class="linenos">37</span></a> <span class="n">where</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;where&quot;</span><span class="p">)</span>
</span><span id="pushdown_predicates-38"><a href="#pushdown_predicates-38"><span class="linenos">38</span></a> <span class="k">if</span> <span class="n">where</span><span class="p">:</span>
</span><span id="pushdown_predicates-39"><a href="#pushdown_predicates-39"><span class="linenos">39</span></a> <span class="n">selected_sources</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span>
</span><span id="pushdown_predicates-40"><a href="#pushdown_predicates-40"><span class="linenos">40</span></a> <span class="n">join_index</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="pushdown_predicates-41"><a href="#pushdown_predicates-41"><span class="linenos">41</span></a> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span> <span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">join</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[])</span>
</span><span id="pushdown_predicates-42"><a href="#pushdown_predicates-42"><span class="linenos">42</span></a> <span class="p">}</span>
</span><span id="pushdown_predicates-43"><a href="#pushdown_predicates-43"><span class="linenos">43</span></a>
</span><span id="pushdown_predicates-44"><a href="#pushdown_predicates-44"><span class="linenos">44</span></a> <span class="c1"># a right join can only push down to itself and not the source FROM table</span>
</span><span id="pushdown_predicates-45"><a href="#pushdown_predicates-45"><span class="linenos">45</span></a> <span class="c1"># presto, trino and athena don&#39;t support inner joins where the RHS is an UNNEST expression</span>
</span><span id="pushdown_predicates-46"><a href="#pushdown_predicates-46"><span class="linenos">46</span></a> <span class="n">pushdown_allowed</span> <span class="o">=</span> <span class="kc">True</span>
</span><span id="pushdown_predicates-47"><a href="#pushdown_predicates-47"><span class="linenos">47</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">source</span><span class="p">)</span> <span class="ow">in</span> <span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="pushdown_predicates-48"><a href="#pushdown_predicates-48"><span class="linenos">48</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
</span><span id="pushdown_predicates-49"><a href="#pushdown_predicates-49"><span class="linenos">49</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
</span><span id="pushdown_predicates-50"><a href="#pushdown_predicates-50"><span class="linenos">50</span></a> <span class="k">if</span> <span class="n">parent</span><span class="o">.</span><span class="n">side</span> <span class="o">==</span> <span class="s2">&quot;RIGHT&quot;</span><span class="p">:</span>
</span><span id="pushdown_predicates-51"><a href="#pushdown_predicates-51"><span class="linenos">51</span></a> <span class="n">selected_sources</span> <span class="o">=</span> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">source</span><span class="p">)}</span>
</span><span id="pushdown_predicates-52"><a href="#pushdown_predicates-52"><span class="linenos">52</span></a> <span class="k">break</span>
</span><span id="pushdown_predicates-53"><a href="#pushdown_predicates-53"><span class="linenos">53</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Unnest</span><span class="p">)</span> <span class="ow">and</span> <span class="n">unnest_requires_cross_join</span><span class="p">:</span>
</span><span id="pushdown_predicates-54"><a href="#pushdown_predicates-54"><span class="linenos">54</span></a> <span class="n">pushdown_allowed</span> <span class="o">=</span> <span class="kc">False</span>
</span><span id="pushdown_predicates-55"><a href="#pushdown_predicates-55"><span class="linenos">55</span></a> <span class="k">break</span>
</span><span id="pushdown_predicates-56"><a href="#pushdown_predicates-56"><span class="linenos">56</span></a>
</span><span id="pushdown_predicates-57"><a href="#pushdown_predicates-57"><span class="linenos">57</span></a> <span class="k">if</span> <span class="n">pushdown_allowed</span><span class="p">:</span>
</span><span id="pushdown_predicates-58"><a href="#pushdown_predicates-58"><span class="linenos">58</span></a> <span class="n">pushdown</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">selected_sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">dialect</span><span class="p">,</span> <span class="n">join_index</span><span class="p">)</span>
</span><span id="pushdown_predicates-59"><a href="#pushdown_predicates-59"><span class="linenos">59</span></a>
</span><span id="pushdown_predicates-60"><a href="#pushdown_predicates-60"><span class="linenos">60</span></a> <span class="c1"># joins should only pushdown into itself, not to other joins</span>
</span><span id="pushdown_predicates-61"><a href="#pushdown_predicates-61"><span class="linenos">61</span></a> <span class="c1"># so we limit the selected sources to only itself</span>
</span><span id="pushdown_predicates-62"><a href="#pushdown_predicates-62"><span class="linenos">62</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;joins&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]:</span>
</span><span id="pushdown_predicates-63"><a href="#pushdown_predicates-63"><span class="linenos">63</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
</span><span id="pushdown_predicates-64"><a href="#pushdown_predicates-64"><span class="linenos">64</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">:</span>
</span><span id="pushdown_predicates-65"><a href="#pushdown_predicates-65"><span class="linenos">65</span></a> <span class="n">pushdown</span><span class="p">(</span>
</span><span id="pushdown_predicates-66"><a href="#pushdown_predicates-66"><span class="linenos">66</span></a> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;on&quot;</span><span class="p">),</span>
</span><span id="pushdown_predicates-67"><a href="#pushdown_predicates-67"><span class="linenos">67</span></a> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">name</span><span class="p">]},</span>
</span><span id="pushdown_predicates-68"><a href="#pushdown_predicates-68"><span class="linenos">68</span></a> <span class="n">scope_ref_count</span><span class="p">,</span>
</span><span id="pushdown_predicates-69"><a href="#pushdown_predicates-69"><span class="linenos">69</span></a> <span class="n">dialect</span><span class="p">,</span>
</span><span id="pushdown_predicates-70"><a href="#pushdown_predicates-70"><span class="linenos">70</span></a> <span class="p">)</span>
</span><span id="pushdown_predicates-71"><a href="#pushdown_predicates-71"><span class="linenos">71</span></a>
</span><span id="pushdown_predicates-72"><a href="#pushdown_predicates-72"><span class="linenos">72</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div>
@ -391,23 +416,23 @@
</div>
<a class="headerlink" href="#pushdown"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown-62"><a href="#pushdown-62"><span class="linenos">62</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">dialect</span><span class="p">,</span> <span class="n">join_index</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="pushdown-63"><a href="#pushdown-63"><span class="linenos">63</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">condition</span><span class="p">:</span>
</span><span id="pushdown-64"><a href="#pushdown-64"><span class="linenos">64</span></a> <span class="k">return</span>
</span><span id="pushdown-65"><a href="#pushdown-65"><span class="linenos">65</span></a>
</span><span id="pushdown-66"><a href="#pushdown-66"><span class="linenos">66</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">condition</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">simplify</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">))</span>
</span><span id="pushdown-67"><a href="#pushdown-67"><span class="linenos">67</span></a> <span class="n">cnf_like</span> <span class="o">=</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">)</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="pushdown-68"><a href="#pushdown-68"><span class="linenos">68</span></a>
</span><span id="pushdown-69"><a href="#pushdown-69"><span class="linenos">69</span></a> <span class="n">predicates</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span>
</span><span id="pushdown-70"><a href="#pushdown-70"><span class="linenos">70</span></a> <span class="n">condition</span><span class="o">.</span><span class="n">flatten</span><span class="p">()</span>
</span><span id="pushdown-71"><a href="#pushdown-71"><span class="linenos">71</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">if</span> <span class="n">cnf_like</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span>
</span><span id="pushdown-72"><a href="#pushdown-72"><span class="linenos">72</span></a> <span class="k">else</span> <span class="p">[</span><span class="n">condition</span><span class="p">]</span>
</span><span id="pushdown-73"><a href="#pushdown-73"><span class="linenos">73</span></a> <span class="p">)</span>
</span><span id="pushdown-74"><a href="#pushdown-74"><span class="linenos">74</span></a>
</span><span id="pushdown-75"><a href="#pushdown-75"><span class="linenos">75</span></a> <span class="k">if</span> <span class="n">cnf_like</span><span class="p">:</span>
</span><span id="pushdown-76"><a href="#pushdown-76"><span class="linenos">76</span></a> <span class="n">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">join_index</span><span class="o">=</span><span class="n">join_index</span><span class="p">)</span>
</span><span id="pushdown-77"><a href="#pushdown-77"><span class="linenos">77</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="pushdown-78"><a href="#pushdown-78"><span class="linenos">78</span></a> <span class="n">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown-75"><a href="#pushdown-75"><span class="linenos">75</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">dialect</span><span class="p">,</span> <span class="n">join_index</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="pushdown-76"><a href="#pushdown-76"><span class="linenos">76</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">condition</span><span class="p">:</span>
</span><span id="pushdown-77"><a href="#pushdown-77"><span class="linenos">77</span></a> <span class="k">return</span>
</span><span id="pushdown-78"><a href="#pushdown-78"><span class="linenos">78</span></a>
</span><span id="pushdown-79"><a href="#pushdown-79"><span class="linenos">79</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">condition</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">simplify</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">))</span>
</span><span id="pushdown-80"><a href="#pushdown-80"><span class="linenos">80</span></a> <span class="n">cnf_like</span> <span class="o">=</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">)</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</span><span id="pushdown-81"><a href="#pushdown-81"><span class="linenos">81</span></a>
</span><span id="pushdown-82"><a href="#pushdown-82"><span class="linenos">82</span></a> <span class="n">predicates</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span>
</span><span id="pushdown-83"><a href="#pushdown-83"><span class="linenos">83</span></a> <span class="n">condition</span><span class="o">.</span><span class="n">flatten</span><span class="p">()</span>
</span><span id="pushdown-84"><a href="#pushdown-84"><span class="linenos">84</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">if</span> <span class="n">cnf_like</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span>
</span><span id="pushdown-85"><a href="#pushdown-85"><span class="linenos">85</span></a> <span class="k">else</span> <span class="p">[</span><span class="n">condition</span><span class="p">]</span>
</span><span id="pushdown-86"><a href="#pushdown-86"><span class="linenos">86</span></a> <span class="p">)</span>
</span><span id="pushdown-87"><a href="#pushdown-87"><span class="linenos">87</span></a>
</span><span id="pushdown-88"><a href="#pushdown-88"><span class="linenos">88</span></a> <span class="k">if</span> <span class="n">cnf_like</span><span class="p">:</span>
</span><span id="pushdown-89"><a href="#pushdown-89"><span class="linenos">89</span></a> <span class="n">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">join_index</span><span class="o">=</span><span class="n">join_index</span><span class="p">)</span>
</span><span id="pushdown-90"><a href="#pushdown-90"><span class="linenos">90</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="pushdown-91"><a href="#pushdown-91"><span class="linenos">91</span></a> <span class="n">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
</span></pre></div>
@ -425,30 +450,30 @@
</div>
<a class="headerlink" href="#pushdown_cnf"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown_cnf-81"><a href="#pushdown_cnf-81"><span class="linenos"> 81</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">join_index</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="pushdown_cnf-82"><a href="#pushdown_cnf-82"><span class="linenos"> 82</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="pushdown_cnf-83"><a href="#pushdown_cnf-83"><span class="linenos"> 83</span></a><span class="sd"> If the predicates are in CNF like form, we can simply replace each block in the parent.</span>
</span><span id="pushdown_cnf-84"><a href="#pushdown_cnf-84"><span class="linenos"> 84</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="pushdown_cnf-85"><a href="#pushdown_cnf-85"><span class="linenos"> 85</span></a> <span class="n">join_index</span> <span class="o">=</span> <span class="n">join_index</span> <span class="ow">or</span> <span class="p">{}</span>
</span><span id="pushdown_cnf-86"><a href="#pushdown_cnf-86"><span class="linenos"> 86</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="pushdown_cnf-87"><a href="#pushdown_cnf-87"><span class="linenos"> 87</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
</span><span id="pushdown_cnf-88"><a href="#pushdown_cnf-88"><span class="linenos"> 88</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
</span><span id="pushdown_cnf-89"><a href="#pushdown_cnf-89"><span class="linenos"> 89</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">alias_or_name</span>
</span><span id="pushdown_cnf-90"><a href="#pushdown_cnf-90"><span class="linenos"> 90</span></a> <span class="n">predicate_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span>
</span><span id="pushdown_cnf-91"><a href="#pushdown_cnf-91"><span class="linenos"> 91</span></a>
</span><span id="pushdown_cnf-92"><a href="#pushdown_cnf-92"><span class="linenos"> 92</span></a> <span class="c1"># Don&#39;t push the predicate if it references tables that appear in later joins</span>
</span><span id="pushdown_cnf-93"><a href="#pushdown_cnf-93"><span class="linenos"> 93</span></a> <span class="n">this_index</span> <span class="o">=</span> <span class="n">join_index</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
</span><span id="pushdown_cnf-94"><a href="#pushdown_cnf-94"><span class="linenos"> 94</span></a> <span class="k">if</span> <span class="nb">all</span><span class="p">(</span><span class="n">join_index</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="o">&lt;</span> <span class="n">this_index</span> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">predicate_tables</span><span class="p">):</span>
</span><span id="pushdown_cnf-95"><a href="#pushdown_cnf-95"><span class="linenos"> 95</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
</span><span id="pushdown_cnf-96"><a href="#pushdown_cnf-96"><span class="linenos"> 96</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="pushdown_cnf-97"><a href="#pushdown_cnf-97"><span class="linenos"> 97</span></a> <span class="k">break</span>
</span><span id="pushdown_cnf-98"><a href="#pushdown_cnf-98"><span class="linenos"> 98</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="pushdown_cnf-99"><a href="#pushdown_cnf-99"><span class="linenos"> 99</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
</span><span id="pushdown_cnf-100"><a href="#pushdown_cnf-100"><span class="linenos">100</span></a> <span class="n">inner_predicate</span> <span class="o">=</span> <span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">)</span>
</span><span id="pushdown_cnf-101"><a href="#pushdown_cnf-101"><span class="linenos">101</span></a> <span class="k">if</span> <span class="n">find_in_scope</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">):</span>
</span><span id="pushdown_cnf-102"><a href="#pushdown_cnf-102"><span class="linenos">102</span></a> <span class="n">node</span><span class="o">.</span><span class="n">having</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="pushdown_cnf-103"><a href="#pushdown_cnf-103"><span class="linenos">103</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="pushdown_cnf-104"><a href="#pushdown_cnf-104"><span class="linenos">104</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown_cnf-94"><a href="#pushdown_cnf-94"><span class="linenos"> 94</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">,</span> <span class="n">join_index</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
</span><span id="pushdown_cnf-95"><a href="#pushdown_cnf-95"><span class="linenos"> 95</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="pushdown_cnf-96"><a href="#pushdown_cnf-96"><span class="linenos"> 96</span></a><span class="sd"> If the predicates are in CNF like form, we can simply replace each block in the parent.</span>
</span><span id="pushdown_cnf-97"><a href="#pushdown_cnf-97"><span class="linenos"> 97</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="pushdown_cnf-98"><a href="#pushdown_cnf-98"><span class="linenos"> 98</span></a> <span class="n">join_index</span> <span class="o">=</span> <span class="n">join_index</span> <span class="ow">or</span> <span class="p">{}</span>
</span><span id="pushdown_cnf-99"><a href="#pushdown_cnf-99"><span class="linenos"> 99</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="pushdown_cnf-100"><a href="#pushdown_cnf-100"><span class="linenos">100</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
</span><span id="pushdown_cnf-101"><a href="#pushdown_cnf-101"><span class="linenos">101</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
</span><span id="pushdown_cnf-102"><a href="#pushdown_cnf-102"><span class="linenos">102</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">alias_or_name</span>
</span><span id="pushdown_cnf-103"><a href="#pushdown_cnf-103"><span class="linenos">103</span></a> <span class="n">predicate_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span>
</span><span id="pushdown_cnf-104"><a href="#pushdown_cnf-104"><span class="linenos">104</span></a>
</span><span id="pushdown_cnf-105"><a href="#pushdown_cnf-105"><span class="linenos">105</span></a> <span class="c1"># Don&#39;t push the predicate if it references tables that appear in later joins</span>
</span><span id="pushdown_cnf-106"><a href="#pushdown_cnf-106"><span class="linenos">106</span></a> <span class="n">this_index</span> <span class="o">=</span> <span class="n">join_index</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
</span><span id="pushdown_cnf-107"><a href="#pushdown_cnf-107"><span class="linenos">107</span></a> <span class="k">if</span> <span class="nb">all</span><span class="p">(</span><span class="n">join_index</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="o">&lt;</span> <span class="n">this_index</span> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">predicate_tables</span><span class="p">):</span>
</span><span id="pushdown_cnf-108"><a href="#pushdown_cnf-108"><span class="linenos">108</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
</span><span id="pushdown_cnf-109"><a href="#pushdown_cnf-109"><span class="linenos">109</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="pushdown_cnf-110"><a href="#pushdown_cnf-110"><span class="linenos">110</span></a> <span class="k">break</span>
</span><span id="pushdown_cnf-111"><a href="#pushdown_cnf-111"><span class="linenos">111</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="pushdown_cnf-112"><a href="#pushdown_cnf-112"><span class="linenos">112</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
</span><span id="pushdown_cnf-113"><a href="#pushdown_cnf-113"><span class="linenos">113</span></a> <span class="n">inner_predicate</span> <span class="o">=</span> <span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">)</span>
</span><span id="pushdown_cnf-114"><a href="#pushdown_cnf-114"><span class="linenos">114</span></a> <span class="k">if</span> <span class="n">find_in_scope</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">):</span>
</span><span id="pushdown_cnf-115"><a href="#pushdown_cnf-115"><span class="linenos">115</span></a> <span class="n">node</span><span class="o">.</span><span class="n">having</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="pushdown_cnf-116"><a href="#pushdown_cnf-116"><span class="linenos">116</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="pushdown_cnf-117"><a href="#pushdown_cnf-117"><span class="linenos">117</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span></pre></div>
@ -468,53 +493,53 @@
</div>
<a class="headerlink" href="#pushdown_dnf"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown_dnf-107"><a href="#pushdown_dnf-107"><span class="linenos">107</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
</span><span id="pushdown_dnf-108"><a href="#pushdown_dnf-108"><span class="linenos">108</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="pushdown_dnf-109"><a href="#pushdown_dnf-109"><span class="linenos">109</span></a><span class="sd"> If the predicates are in DNF form, we can only push down conditions that are in all blocks.</span>
</span><span id="pushdown_dnf-110"><a href="#pushdown_dnf-110"><span class="linenos">110</span></a><span class="sd"> Additionally, we can&#39;t remove predicates from their original form.</span>
</span><span id="pushdown_dnf-111"><a href="#pushdown_dnf-111"><span class="linenos">111</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="pushdown_dnf-112"><a href="#pushdown_dnf-112"><span class="linenos">112</span></a> <span class="c1"># find all the tables that can be pushdown too</span>
</span><span id="pushdown_dnf-113"><a href="#pushdown_dnf-113"><span class="linenos">113</span></a> <span class="c1"># these are tables that are referenced in all blocks of a DNF</span>
</span><span id="pushdown_dnf-114"><a href="#pushdown_dnf-114"><span class="linenos">114</span></a> <span class="c1"># (a.x AND b.x) OR (a.y AND c.y)</span>
</span><span id="pushdown_dnf-115"><a href="#pushdown_dnf-115"><span class="linenos">115</span></a> <span class="c1"># only table a can be push down</span>
</span><span id="pushdown_dnf-116"><a href="#pushdown_dnf-116"><span class="linenos">116</span></a> <span class="n">pushdown_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
</span><span id="pushdown_dnf-117"><a href="#pushdown_dnf-117"><span class="linenos">117</span></a>
</span><span id="pushdown_dnf-118"><a href="#pushdown_dnf-118"><span class="linenos">118</span></a> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="pushdown_dnf-119"><a href="#pushdown_dnf-119"><span class="linenos">119</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
</span><span id="pushdown_dnf-120"><a href="#pushdown_dnf-120"><span class="linenos">120</span></a>
</span><span id="pushdown_dnf-121"><a href="#pushdown_dnf-121"><span class="linenos">121</span></a> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="pushdown_dnf-122"><a href="#pushdown_dnf-122"><span class="linenos">122</span></a> <span class="n">a_tables</span> <span class="o">&amp;=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
</span><span id="pushdown_dnf-123"><a href="#pushdown_dnf-123"><span class="linenos">123</span></a>
</span><span id="pushdown_dnf-124"><a href="#pushdown_dnf-124"><span class="linenos">124</span></a> <span class="n">pushdown_tables</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">a_tables</span><span class="p">)</span>
</span><span id="pushdown_dnf-125"><a href="#pushdown_dnf-125"><span class="linenos">125</span></a>
</span><span id="pushdown_dnf-126"><a href="#pushdown_dnf-126"><span class="linenos">126</span></a> <span class="n">conditions</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="pushdown_dnf-127"><a href="#pushdown_dnf-127"><span class="linenos">127</span></a>
</span><span id="pushdown_dnf-128"><a href="#pushdown_dnf-128"><span class="linenos">128</span></a> <span class="c1"># pushdown all predicates to their respective nodes</span>
</span><span id="pushdown_dnf-129"><a href="#pushdown_dnf-129"><span class="linenos">129</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">pushdown_tables</span><span class="p">):</span>
</span><span id="pushdown_dnf-130"><a href="#pushdown_dnf-130"><span class="linenos">130</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="pushdown_dnf-131"><a href="#pushdown_dnf-131"><span class="linenos">131</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
</span><span id="pushdown_dnf-132"><a href="#pushdown_dnf-132"><span class="linenos">132</span></a>
</span><span id="pushdown_dnf-133"><a href="#pushdown_dnf-133"><span class="linenos">133</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">nodes</span><span class="p">:</span>
</span><span id="pushdown_dnf-134"><a href="#pushdown_dnf-134"><span class="linenos">134</span></a> <span class="k">continue</span>
</span><span id="pushdown_dnf-135"><a href="#pushdown_dnf-135"><span class="linenos">135</span></a>
</span><span id="pushdown_dnf-136"><a href="#pushdown_dnf-136"><span class="linenos">136</span></a> <span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span>
</span><span id="pushdown_dnf-137"><a href="#pushdown_dnf-137"><span class="linenos">137</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span><span class="p">(</span><span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">],</span> <span class="n">predicate</span><span class="p">)</span> <span class="k">if</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">conditions</span> <span class="k">else</span> <span class="n">predicate</span>
</span><span id="pushdown_dnf-138"><a href="#pushdown_dnf-138"><span class="linenos">138</span></a> <span class="p">)</span>
</span><span id="pushdown_dnf-139"><a href="#pushdown_dnf-139"><span class="linenos">139</span></a>
</span><span id="pushdown_dnf-140"><a href="#pushdown_dnf-140"><span class="linenos">140</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="pushdown_dnf-141"><a href="#pushdown_dnf-141"><span class="linenos">141</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">conditions</span><span class="p">:</span>
</span><span id="pushdown_dnf-142"><a href="#pushdown_dnf-142"><span class="linenos">142</span></a> <span class="k">continue</span>
</span><span id="pushdown_dnf-143"><a href="#pushdown_dnf-143"><span class="linenos">143</span></a>
</span><span id="pushdown_dnf-144"><a href="#pushdown_dnf-144"><span class="linenos">144</span></a> <span class="n">predicate</span> <span class="o">=</span> <span class="n">conditions</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown_dnf-120"><a href="#pushdown_dnf-120"><span class="linenos">120</span></a><span class="k">def</span><span class="w"> </span><span class="nf">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
</span><span id="pushdown_dnf-121"><a href="#pushdown_dnf-121"><span class="linenos">121</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="pushdown_dnf-122"><a href="#pushdown_dnf-122"><span class="linenos">122</span></a><span class="sd"> If the predicates are in DNF form, we can only push down conditions that are in all blocks.</span>
</span><span id="pushdown_dnf-123"><a href="#pushdown_dnf-123"><span class="linenos">123</span></a><span class="sd"> Additionally, we can&#39;t remove predicates from their original form.</span>
</span><span id="pushdown_dnf-124"><a href="#pushdown_dnf-124"><span class="linenos">124</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="pushdown_dnf-125"><a href="#pushdown_dnf-125"><span class="linenos">125</span></a> <span class="c1"># find all the tables that can be pushdown too</span>
</span><span id="pushdown_dnf-126"><a href="#pushdown_dnf-126"><span class="linenos">126</span></a> <span class="c1"># these are tables that are referenced in all blocks of a DNF</span>
</span><span id="pushdown_dnf-127"><a href="#pushdown_dnf-127"><span class="linenos">127</span></a> <span class="c1"># (a.x AND b.x) OR (a.y AND c.y)</span>
</span><span id="pushdown_dnf-128"><a href="#pushdown_dnf-128"><span class="linenos">128</span></a> <span class="c1"># only table a can be push down</span>
</span><span id="pushdown_dnf-129"><a href="#pushdown_dnf-129"><span class="linenos">129</span></a> <span class="n">pushdown_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
</span><span id="pushdown_dnf-130"><a href="#pushdown_dnf-130"><span class="linenos">130</span></a>
</span><span id="pushdown_dnf-131"><a href="#pushdown_dnf-131"><span class="linenos">131</span></a> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="pushdown_dnf-132"><a href="#pushdown_dnf-132"><span class="linenos">132</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
</span><span id="pushdown_dnf-133"><a href="#pushdown_dnf-133"><span class="linenos">133</span></a>
</span><span id="pushdown_dnf-134"><a href="#pushdown_dnf-134"><span class="linenos">134</span></a> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="pushdown_dnf-135"><a href="#pushdown_dnf-135"><span class="linenos">135</span></a> <span class="n">a_tables</span> <span class="o">&amp;=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
</span><span id="pushdown_dnf-136"><a href="#pushdown_dnf-136"><span class="linenos">136</span></a>
</span><span id="pushdown_dnf-137"><a href="#pushdown_dnf-137"><span class="linenos">137</span></a> <span class="n">pushdown_tables</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">a_tables</span><span class="p">)</span>
</span><span id="pushdown_dnf-138"><a href="#pushdown_dnf-138"><span class="linenos">138</span></a>
</span><span id="pushdown_dnf-139"><a href="#pushdown_dnf-139"><span class="linenos">139</span></a> <span class="n">conditions</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="pushdown_dnf-140"><a href="#pushdown_dnf-140"><span class="linenos">140</span></a>
</span><span id="pushdown_dnf-141"><a href="#pushdown_dnf-141"><span class="linenos">141</span></a> <span class="c1"># pushdown all predicates to their respective nodes</span>
</span><span id="pushdown_dnf-142"><a href="#pushdown_dnf-142"><span class="linenos">142</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">pushdown_tables</span><span class="p">):</span>
</span><span id="pushdown_dnf-143"><a href="#pushdown_dnf-143"><span class="linenos">143</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
</span><span id="pushdown_dnf-144"><a href="#pushdown_dnf-144"><span class="linenos">144</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
</span><span id="pushdown_dnf-145"><a href="#pushdown_dnf-145"><span class="linenos">145</span></a>
</span><span id="pushdown_dnf-146"><a href="#pushdown_dnf-146"><span class="linenos">146</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
</span><span id="pushdown_dnf-147"><a href="#pushdown_dnf-147"><span class="linenos">147</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="pushdown_dnf-148"><a href="#pushdown_dnf-148"><span class="linenos">148</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="pushdown_dnf-149"><a href="#pushdown_dnf-149"><span class="linenos">149</span></a> <span class="n">inner_predicate</span> <span class="o">=</span> <span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">)</span>
</span><span id="pushdown_dnf-150"><a href="#pushdown_dnf-150"><span class="linenos">150</span></a> <span class="k">if</span> <span class="n">find_in_scope</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">):</span>
</span><span id="pushdown_dnf-151"><a href="#pushdown_dnf-151"><span class="linenos">151</span></a> <span class="n">node</span><span class="o">.</span><span class="n">having</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="pushdown_dnf-152"><a href="#pushdown_dnf-152"><span class="linenos">152</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="pushdown_dnf-153"><a href="#pushdown_dnf-153"><span class="linenos">153</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="pushdown_dnf-146"><a href="#pushdown_dnf-146"><span class="linenos">146</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">nodes</span><span class="p">:</span>
</span><span id="pushdown_dnf-147"><a href="#pushdown_dnf-147"><span class="linenos">147</span></a> <span class="k">continue</span>
</span><span id="pushdown_dnf-148"><a href="#pushdown_dnf-148"><span class="linenos">148</span></a>
</span><span id="pushdown_dnf-149"><a href="#pushdown_dnf-149"><span class="linenos">149</span></a> <span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span>
</span><span id="pushdown_dnf-150"><a href="#pushdown_dnf-150"><span class="linenos">150</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span><span class="p">(</span><span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">],</span> <span class="n">predicate</span><span class="p">)</span> <span class="k">if</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">conditions</span> <span class="k">else</span> <span class="n">predicate</span>
</span><span id="pushdown_dnf-151"><a href="#pushdown_dnf-151"><span class="linenos">151</span></a> <span class="p">)</span>
</span><span id="pushdown_dnf-152"><a href="#pushdown_dnf-152"><span class="linenos">152</span></a>
</span><span id="pushdown_dnf-153"><a href="#pushdown_dnf-153"><span class="linenos">153</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
</span><span id="pushdown_dnf-154"><a href="#pushdown_dnf-154"><span class="linenos">154</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">conditions</span><span class="p">:</span>
</span><span id="pushdown_dnf-155"><a href="#pushdown_dnf-155"><span class="linenos">155</span></a> <span class="k">continue</span>
</span><span id="pushdown_dnf-156"><a href="#pushdown_dnf-156"><span class="linenos">156</span></a>
</span><span id="pushdown_dnf-157"><a href="#pushdown_dnf-157"><span class="linenos">157</span></a> <span class="n">predicate</span> <span class="o">=</span> <span class="n">conditions</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
</span><span id="pushdown_dnf-158"><a href="#pushdown_dnf-158"><span class="linenos">158</span></a>
</span><span id="pushdown_dnf-159"><a href="#pushdown_dnf-159"><span class="linenos">159</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
</span><span id="pushdown_dnf-160"><a href="#pushdown_dnf-160"><span class="linenos">160</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="pushdown_dnf-161"><a href="#pushdown_dnf-161"><span class="linenos">161</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="pushdown_dnf-162"><a href="#pushdown_dnf-162"><span class="linenos">162</span></a> <span class="n">inner_predicate</span> <span class="o">=</span> <span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">)</span>
</span><span id="pushdown_dnf-163"><a href="#pushdown_dnf-163"><span class="linenos">163</span></a> <span class="k">if</span> <span class="n">find_in_scope</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">):</span>
</span><span id="pushdown_dnf-164"><a href="#pushdown_dnf-164"><span class="linenos">164</span></a> <span class="n">node</span><span class="o">.</span><span class="n">having</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="pushdown_dnf-165"><a href="#pushdown_dnf-165"><span class="linenos">165</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="pushdown_dnf-166"><a href="#pushdown_dnf-166"><span class="linenos">166</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">inner_predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span></pre></div>
@ -535,44 +560,44 @@ Additionally, we can't remove predicates from their original form.</p>
</div>
<a class="headerlink" href="#nodes_for_predicate"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="nodes_for_predicate-156"><a href="#nodes_for_predicate-156"><span class="linenos">156</span></a><span class="k">def</span><span class="w"> </span><span class="nf">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
</span><span id="nodes_for_predicate-157"><a href="#nodes_for_predicate-157"><span class="linenos">157</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="nodes_for_predicate-158"><a href="#nodes_for_predicate-158"><span class="linenos">158</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
</span><span id="nodes_for_predicate-159"><a href="#nodes_for_predicate-159"><span class="linenos">159</span></a> <span class="n">where_condition</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">)</span>
</span><span id="nodes_for_predicate-160"><a href="#nodes_for_predicate-160"><span class="linenos">160</span></a>
</span><span id="nodes_for_predicate-161"><a href="#nodes_for_predicate-161"><span class="linenos">161</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">tables</span><span class="p">):</span>
</span><span id="nodes_for_predicate-162"><a href="#nodes_for_predicate-162"><span class="linenos">162</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">source</span> <span class="o">=</span> <span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="ow">or</span> <span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="nodes_for_predicate-163"><a href="#nodes_for_predicate-163"><span class="linenos">163</span></a>
</span><span id="nodes_for_predicate-164"><a href="#nodes_for_predicate-164"><span class="linenos">164</span></a> <span class="c1"># if the predicate is in a where statement we can try to push it down</span>
</span><span id="nodes_for_predicate-165"><a href="#nodes_for_predicate-165"><span class="linenos">165</span></a> <span class="c1"># we want to find the root join or from statement</span>
</span><span id="nodes_for_predicate-166"><a href="#nodes_for_predicate-166"><span class="linenos">166</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">where_condition</span><span class="p">:</span>
</span><span id="nodes_for_predicate-167"><a href="#nodes_for_predicate-167"><span class="linenos">167</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
</span><span id="nodes_for_predicate-168"><a href="#nodes_for_predicate-168"><span class="linenos">168</span></a>
</span><span id="nodes_for_predicate-169"><a href="#nodes_for_predicate-169"><span class="linenos">169</span></a> <span class="c1"># a node can reference a CTE which should be pushed down</span>
</span><span id="nodes_for_predicate-170"><a href="#nodes_for_predicate-170"><span class="linenos">170</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="nodes_for_predicate-171"><a href="#nodes_for_predicate-171"><span class="linenos">171</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">)</span>
</span><span id="nodes_for_predicate-172"><a href="#nodes_for_predicate-172"><span class="linenos">172</span></a> <span class="k">if</span> <span class="n">with_</span> <span class="ow">and</span> <span class="n">with_</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
</span><span id="nodes_for_predicate-173"><a href="#nodes_for_predicate-173"><span class="linenos">173</span></a> <span class="k">return</span> <span class="p">{}</span>
</span><span id="nodes_for_predicate-174"><a href="#nodes_for_predicate-174"><span class="linenos">174</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="nodes_for_predicate-175"><a href="#nodes_for_predicate-175"><span class="linenos">175</span></a>
</span><span id="nodes_for_predicate-176"><a href="#nodes_for_predicate-176"><span class="linenos">176</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
</span><span id="nodes_for_predicate-177"><a href="#nodes_for_predicate-177"><span class="linenos">177</span></a> <span class="k">if</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="o">!=</span> <span class="s2">&quot;RIGHT&quot;</span><span class="p">:</span>
</span><span id="nodes_for_predicate-178"><a href="#nodes_for_predicate-178"><span class="linenos">178</span></a> <span class="k">return</span> <span class="p">{}</span>
</span><span id="nodes_for_predicate-179"><a href="#nodes_for_predicate-179"><span class="linenos">179</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
</span><span id="nodes_for_predicate-180"><a href="#nodes_for_predicate-180"><span class="linenos">180</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
</span><span id="nodes_for_predicate-181"><a href="#nodes_for_predicate-181"><span class="linenos">181</span></a> <span class="c1"># We can&#39;t push down window expressions</span>
</span><span id="nodes_for_predicate-182"><a href="#nodes_for_predicate-182"><span class="linenos">182</span></a> <span class="n">has_window_expression</span> <span class="o">=</span> <span class="nb">any</span><span class="p">(</span>
</span><span id="nodes_for_predicate-183"><a href="#nodes_for_predicate-183"><span class="linenos">183</span></a> <span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
</span><span id="nodes_for_predicate-184"><a href="#nodes_for_predicate-184"><span class="linenos">184</span></a> <span class="p">)</span>
</span><span id="nodes_for_predicate-185"><a href="#nodes_for_predicate-185"><span class="linenos">185</span></a> <span class="c1"># we can&#39;t push down predicates to select statements if they are referenced in</span>
</span><span id="nodes_for_predicate-186"><a href="#nodes_for_predicate-186"><span class="linenos">186</span></a> <span class="c1"># multiple places.</span>
</span><span id="nodes_for_predicate-187"><a href="#nodes_for_predicate-187"><span class="linenos">187</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="nodes_for_predicate-188"><a href="#nodes_for_predicate-188"><span class="linenos">188</span></a> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;group&quot;</span><span class="p">)</span>
</span><span id="nodes_for_predicate-189"><a href="#nodes_for_predicate-189"><span class="linenos">189</span></a> <span class="ow">and</span> <span class="n">scope_ref_count</span><span class="p">[</span><span class="nb">id</span><span class="p">(</span><span class="n">source</span><span class="p">)]</span> <span class="o">&lt;</span> <span class="mi">2</span>
</span><span id="nodes_for_predicate-190"><a href="#nodes_for_predicate-190"><span class="linenos">190</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">has_window_expression</span>
</span><span id="nodes_for_predicate-191"><a href="#nodes_for_predicate-191"><span class="linenos">191</span></a> <span class="p">):</span>
</span><span id="nodes_for_predicate-192"><a href="#nodes_for_predicate-192"><span class="linenos">192</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
</span><span id="nodes_for_predicate-193"><a href="#nodes_for_predicate-193"><span class="linenos">193</span></a> <span class="k">return</span> <span class="n">nodes</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="nodes_for_predicate-169"><a href="#nodes_for_predicate-169"><span class="linenos">169</span></a><span class="k">def</span><span class="w"> </span><span class="nf">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
</span><span id="nodes_for_predicate-170"><a href="#nodes_for_predicate-170"><span class="linenos">170</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="nodes_for_predicate-171"><a href="#nodes_for_predicate-171"><span class="linenos">171</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
</span><span id="nodes_for_predicate-172"><a href="#nodes_for_predicate-172"><span class="linenos">172</span></a> <span class="n">where_condition</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">)</span>
</span><span id="nodes_for_predicate-173"><a href="#nodes_for_predicate-173"><span class="linenos">173</span></a>
</span><span id="nodes_for_predicate-174"><a href="#nodes_for_predicate-174"><span class="linenos">174</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">tables</span><span class="p">):</span>
</span><span id="nodes_for_predicate-175"><a href="#nodes_for_predicate-175"><span class="linenos">175</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">source</span> <span class="o">=</span> <span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="ow">or</span> <span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
</span><span id="nodes_for_predicate-176"><a href="#nodes_for_predicate-176"><span class="linenos">176</span></a>
</span><span id="nodes_for_predicate-177"><a href="#nodes_for_predicate-177"><span class="linenos">177</span></a> <span class="c1"># if the predicate is in a where statement we can try to push it down</span>
</span><span id="nodes_for_predicate-178"><a href="#nodes_for_predicate-178"><span class="linenos">178</span></a> <span class="c1"># we want to find the root join or from statement</span>
</span><span id="nodes_for_predicate-179"><a href="#nodes_for_predicate-179"><span class="linenos">179</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">where_condition</span><span class="p">:</span>
</span><span id="nodes_for_predicate-180"><a href="#nodes_for_predicate-180"><span class="linenos">180</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
</span><span id="nodes_for_predicate-181"><a href="#nodes_for_predicate-181"><span class="linenos">181</span></a>
</span><span id="nodes_for_predicate-182"><a href="#nodes_for_predicate-182"><span class="linenos">182</span></a> <span class="c1"># a node can reference a CTE which should be pushed down</span>
</span><span id="nodes_for_predicate-183"><a href="#nodes_for_predicate-183"><span class="linenos">183</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="nodes_for_predicate-184"><a href="#nodes_for_predicate-184"><span class="linenos">184</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;with&quot;</span><span class="p">)</span>
</span><span id="nodes_for_predicate-185"><a href="#nodes_for_predicate-185"><span class="linenos">185</span></a> <span class="k">if</span> <span class="n">with_</span> <span class="ow">and</span> <span class="n">with_</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
</span><span id="nodes_for_predicate-186"><a href="#nodes_for_predicate-186"><span class="linenos">186</span></a> <span class="k">return</span> <span class="p">{}</span>
</span><span id="nodes_for_predicate-187"><a href="#nodes_for_predicate-187"><span class="linenos">187</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
</span><span id="nodes_for_predicate-188"><a href="#nodes_for_predicate-188"><span class="linenos">188</span></a>
</span><span id="nodes_for_predicate-189"><a href="#nodes_for_predicate-189"><span class="linenos">189</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
</span><span id="nodes_for_predicate-190"><a href="#nodes_for_predicate-190"><span class="linenos">190</span></a> <span class="k">if</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="o">!=</span> <span class="s2">&quot;RIGHT&quot;</span><span class="p">:</span>
</span><span id="nodes_for_predicate-191"><a href="#nodes_for_predicate-191"><span class="linenos">191</span></a> <span class="k">return</span> <span class="p">{}</span>
</span><span id="nodes_for_predicate-192"><a href="#nodes_for_predicate-192"><span class="linenos">192</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
</span><span id="nodes_for_predicate-193"><a href="#nodes_for_predicate-193"><span class="linenos">193</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
</span><span id="nodes_for_predicate-194"><a href="#nodes_for_predicate-194"><span class="linenos">194</span></a> <span class="c1"># We can&#39;t push down window expressions</span>
</span><span id="nodes_for_predicate-195"><a href="#nodes_for_predicate-195"><span class="linenos">195</span></a> <span class="n">has_window_expression</span> <span class="o">=</span> <span class="nb">any</span><span class="p">(</span>
</span><span id="nodes_for_predicate-196"><a href="#nodes_for_predicate-196"><span class="linenos">196</span></a> <span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
</span><span id="nodes_for_predicate-197"><a href="#nodes_for_predicate-197"><span class="linenos">197</span></a> <span class="p">)</span>
</span><span id="nodes_for_predicate-198"><a href="#nodes_for_predicate-198"><span class="linenos">198</span></a> <span class="c1"># we can&#39;t push down predicates to select statements if they are referenced in</span>
</span><span id="nodes_for_predicate-199"><a href="#nodes_for_predicate-199"><span class="linenos">199</span></a> <span class="c1"># multiple places.</span>
</span><span id="nodes_for_predicate-200"><a href="#nodes_for_predicate-200"><span class="linenos">200</span></a> <span class="k">if</span> <span class="p">(</span>
</span><span id="nodes_for_predicate-201"><a href="#nodes_for_predicate-201"><span class="linenos">201</span></a> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;group&quot;</span><span class="p">)</span>
</span><span id="nodes_for_predicate-202"><a href="#nodes_for_predicate-202"><span class="linenos">202</span></a> <span class="ow">and</span> <span class="n">scope_ref_count</span><span class="p">[</span><span class="nb">id</span><span class="p">(</span><span class="n">source</span><span class="p">)]</span> <span class="o">&lt;</span> <span class="mi">2</span>
</span><span id="nodes_for_predicate-203"><a href="#nodes_for_predicate-203"><span class="linenos">203</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">has_window_expression</span>
</span><span id="nodes_for_predicate-204"><a href="#nodes_for_predicate-204"><span class="linenos">204</span></a> <span class="p">):</span>
</span><span id="nodes_for_predicate-205"><a href="#nodes_for_predicate-205"><span class="linenos">205</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
</span><span id="nodes_for_predicate-206"><a href="#nodes_for_predicate-206"><span class="linenos">206</span></a> <span class="k">return</span> <span class="n">nodes</span>
</span></pre></div>
@ -590,21 +615,21 @@ Additionally, we can't remove predicates from their original form.</p>
</div>
<a class="headerlink" href="#replace_aliases"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="replace_aliases-196"><a href="#replace_aliases-196"><span class="linenos">196</span></a><span class="k">def</span><span class="w"> </span><span class="nf">replace_aliases</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">predicate</span><span class="p">):</span>
</span><span id="replace_aliases-197"><a href="#replace_aliases-197"><span class="linenos">197</span></a> <span class="n">aliases</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="replace_aliases-198"><a href="#replace_aliases-198"><span class="linenos">198</span></a>
</span><span id="replace_aliases-199"><a href="#replace_aliases-199"><span class="linenos">199</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
</span><span id="replace_aliases-200"><a href="#replace_aliases-200"><span class="linenos">200</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
</span><span id="replace_aliases-201"><a href="#replace_aliases-201"><span class="linenos">201</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
</span><span id="replace_aliases-202"><a href="#replace_aliases-202"><span class="linenos">202</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="replace_aliases-203"><a href="#replace_aliases-203"><span class="linenos">203</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span>
</span><span id="replace_aliases-204"><a href="#replace_aliases-204"><span class="linenos">204</span></a>
</span><span id="replace_aliases-205"><a href="#replace_aliases-205"><span class="linenos">205</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">_replace_alias</span><span class="p">(</span><span class="n">column</span><span class="p">):</span>
</span><span id="replace_aliases-206"><a href="#replace_aliases-206"><span class="linenos">206</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="ow">and</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">aliases</span><span class="p">:</span>
</span><span id="replace_aliases-207"><a href="#replace_aliases-207"><span class="linenos">207</span></a> <span class="k">return</span> <span class="n">aliases</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
</span><span id="replace_aliases-208"><a href="#replace_aliases-208"><span class="linenos">208</span></a> <span class="k">return</span> <span class="n">column</span>
</span><span id="replace_aliases-209"><a href="#replace_aliases-209"><span class="linenos">209</span></a>
</span><span id="replace_aliases-210"><a href="#replace_aliases-210"><span class="linenos">210</span></a> <span class="k">return</span> <span class="n">predicate</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_replace_alias</span><span class="p">)</span>
<div class="pdoc-code codehilite"><pre><span></span><span id="replace_aliases-209"><a href="#replace_aliases-209"><span class="linenos">209</span></a><span class="k">def</span><span class="w"> </span><span class="nf">replace_aliases</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">predicate</span><span class="p">):</span>
</span><span id="replace_aliases-210"><a href="#replace_aliases-210"><span class="linenos">210</span></a> <span class="n">aliases</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="replace_aliases-211"><a href="#replace_aliases-211"><span class="linenos">211</span></a>
</span><span id="replace_aliases-212"><a href="#replace_aliases-212"><span class="linenos">212</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
</span><span id="replace_aliases-213"><a href="#replace_aliases-213"><span class="linenos">213</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
</span><span id="replace_aliases-214"><a href="#replace_aliases-214"><span class="linenos">214</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
</span><span id="replace_aliases-215"><a href="#replace_aliases-215"><span class="linenos">215</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="replace_aliases-216"><a href="#replace_aliases-216"><span class="linenos">216</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span>
</span><span id="replace_aliases-217"><a href="#replace_aliases-217"><span class="linenos">217</span></a>
</span><span id="replace_aliases-218"><a href="#replace_aliases-218"><span class="linenos">218</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">_replace_alias</span><span class="p">(</span><span class="n">column</span><span class="p">):</span>
</span><span id="replace_aliases-219"><a href="#replace_aliases-219"><span class="linenos">219</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="ow">and</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">aliases</span><span class="p">:</span>
</span><span id="replace_aliases-220"><a href="#replace_aliases-220"><span class="linenos">220</span></a> <span class="k">return</span> <span class="n">aliases</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
</span><span id="replace_aliases-221"><a href="#replace_aliases-221"><span class="linenos">221</span></a> <span class="k">return</span> <span class="n">column</span>
</span><span id="replace_aliases-222"><a href="#replace_aliases-222"><span class="linenos">222</span></a>
</span><span id="replace_aliases-223"><a href="#replace_aliases-223"><span class="linenos">223</span></a> <span class="k">return</span> <span class="n">predicate</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_replace_alias</span><span class="p">)</span>
</span></pre></div>

4
docs/sqlglot/optimizer/simplify.html
View file

@ -3186,7 +3186,7 @@ prefix are statically known.</p>
<div class="attr variable">
<span class="name">DATETRUNC_COMPARISONS</span> =
<input id="DATETRUNC_COMPARISONS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="../expressions.html#In">sqlglot.expressions.In</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;}</span>
<label class="view-value-button pdoc-button" for="DATETRUNC_COMPARISONS-view-value"></label><span class="default_value">{&lt;class &#39;<a href="../expressions.html#GTE">sqlglot.expressions.GTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#In">sqlglot.expressions.In</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LTE">sqlglot.expressions.LTE</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#EQ">sqlglot.expressions.EQ</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#GT">sqlglot.expressions.GT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#LT">sqlglot.expressions.LT</a>&#39;&gt;, &lt;class &#39;<a href="../expressions.html#NEQ">sqlglot.expressions.NEQ</a>&#39;&gt;}</span>
</div>
@ -3270,7 +3270,7 @@ prefix are statically known.</p>
<section id="JOINS">
<div class="attr variable">
<span class="name">JOINS</span> =
<span class="default_value">{(&#39;&#39;, &#39;INNER&#39;), (&#39;RIGHT&#39;, &#39;&#39;), (&#39;RIGHT&#39;, &#39;OUTER&#39;), (&#39;&#39;, &#39;&#39;)}</span>
<span class="default_value">{(&#39;&#39;, &#39;&#39;), (&#39;RIGHT&#39;, &#39;OUTER&#39;), (&#39;&#39;, &#39;INNER&#39;), (&#39;RIGHT&#39;, &#39;&#39;)}</span>
</div>

28521
docs/sqlglot/parser.html
View file

File diff suppressed because one or more lines are too long

2
docs/sqlglot/time.html
View file

File diff suppressed because one or more lines are too long

2486
docs/sqlglot/tokens.html
View file

File diff suppressed because one or more lines are too long

3282
docs/sqlglot/transforms.html
View file

File diff suppressed because it is too large Load diff

22
sqlglot/dialects/bigquery.py
View file

@ -5,6 +5,7 @@ import re
import typing as t
from sqlglot import exp, generator, parser, tokens, transforms
from sqlglot._typing import E
from sqlglot.dialects.dialect import (
Dialect,
NormalizationStrategy,
@ -35,7 +36,7 @@ from sqlglot.tokens import TokenType
from sqlglot.generator import unsupported_args
if t.TYPE_CHECKING:
from sqlglot._typing import E, Lit
from sqlglot._typing import Lit
from sqlglot.optimizer.annotate_types import TypeAnnotator
@ -352,6 +353,17 @@ def _json_extract_sql(self: BigQuery.Generator, expression: JSON_EXTRACT_TYPE) -
return sql
def _annotate_concat(self: TypeAnnotator, expression: exp.Concat) -> exp.Concat:
annotated = self._annotate_by_args(expression, "expressions")
# Args must be BYTES or types that can be cast to STRING, return type is either BYTES or STRING
# https://cloud.google.com/bigquery/docs/reference/standard-sql/string_functions#concat
if not annotated.is_type(exp.DataType.Type.BINARY, exp.DataType.Type.UNKNOWN):
annotated.type = exp.DataType.Type.VARCHAR
return annotated
class BigQuery(Dialect):
WEEK_OFFSET = -1
UNNEST_COLUMN_ONLY = True
@ -432,7 +444,7 @@ class BigQuery(Dialect):
exp.Substring,
)
},
exp.Concat: lambda self, e: self._annotate_by_args(e, "expressions"),
exp.Concat: _annotate_concat,
exp.Sign: lambda self, e: self._annotate_by_args(e, "this"),
exp.Split: lambda self, e: self._annotate_by_args(e, "this", array=True),
}
@ -440,7 +452,7 @@ class BigQuery(Dialect):
def normalize_identifier(self, expression: E) -> E:
if (
isinstance(expression, exp.Identifier)
and self.normalization_strategy is not NormalizationStrategy.CASE_SENSITIVE
and self.normalization_strategy is NormalizationStrategy.CASE_INSENSITIVE
):
parent = expression.parent
while isinstance(parent, exp.Dot):
@ -462,7 +474,9 @@ class BigQuery(Dialect):
if not case_sensitive:
expression.set("this", expression.this.lower())
return expression
return t.cast(E, expression)
return super().normalize_identifier(expression)
class Tokenizer(tokens.Tokenizer):
QUOTES = ["'", '"', '"""', "'''"]

1
sqlglot/dialects/clickhouse.py
View file

@ -253,6 +253,7 @@ class ClickHouse(Dialect):
"DYNAMIC": TokenType.DYNAMIC,
"ENUM8": TokenType.ENUM8,
"ENUM16": TokenType.ENUM16,
"EXCHANGE": TokenType.COMMAND,
"FINAL": TokenType.FINAL,
"FIXEDSTRING": TokenType.FIXEDSTRING,
"FLOAT32": TokenType.FLOAT,

13
sqlglot/dialects/databricks.py
View file

@ -1,5 +1,7 @@
from __future__ import annotations
from copy import deepcopy
from collections import defaultdict
from sqlglot import exp, transforms, jsonpath
from sqlglot.dialects.dialect import (
@ -10,6 +12,7 @@ from sqlglot.dialects.dialect import (
)
from sqlglot.dialects.spark import Spark
from sqlglot.tokens import TokenType
from sqlglot.optimizer.annotate_types import TypeAnnotator
def _jsonextract_sql(
@ -24,6 +27,16 @@ class Databricks(Spark):
SAFE_DIVISION = False
COPY_PARAMS_ARE_CSV = False
COERCES_TO = defaultdict(set, deepcopy(TypeAnnotator.COERCES_TO))
for text_type in exp.DataType.TEXT_TYPES:
COERCES_TO[text_type] |= {
*exp.DataType.NUMERIC_TYPES,
*exp.DataType.TEMPORAL_TYPES,
exp.DataType.Type.BINARY,
exp.DataType.Type.BOOLEAN,
exp.DataType.Type.INTERVAL,
}
class JSONPathTokenizer(jsonpath.JSONPathTokenizer):
IDENTIFIERS = ["`", '"']

5
sqlglot/dialects/dialect.py
View file

@ -239,7 +239,7 @@ class _Dialect(type):
if enum not in ("", "bigquery"):
klass.generator_class.SELECT_KINDS = ()
if enum not in ("", "athena", "presto", "trino"):
if enum not in ("", "athena", "presto", "trino", "duckdb"):
klass.generator_class.TRY_SUPPORTED = False
klass.generator_class.SUPPORTS_UESCAPE = False
@ -789,6 +789,9 @@ class Dialect(metaclass=_Dialect):
exp.VarMap: lambda self, e: self._annotate_map(e),
}
# Specifies what types a given type can be coerced into
COERCES_TO: t.Dict[exp.DataType.Type, t.Set[exp.DataType.Type]] = {}
@classmethod
def get_or_raise(cls, dialect: DialectType) -> Dialect:
"""

1
sqlglot/dialects/doris.py
View file

@ -59,6 +59,7 @@ class Doris(MySQL):
exp.ArgMax: rename_func("MAX_BY"),
exp.ArgMin: rename_func("MIN_BY"),
exp.ArrayAgg: rename_func("COLLECT_LIST"),
exp.ArrayToString: rename_func("ARRAY_JOIN"),
exp.ArrayUniqueAgg: rename_func("COLLECT_SET"),
exp.CurrentTimestamp: lambda self, _: self.func("NOW"),
exp.DateTrunc: lambda self, e: self.func("DATE_TRUNC", e.this, unit_to_str(e)),

8
sqlglot/dialects/druid.py
View file

@ -1,5 +1,5 @@
from sqlglot import exp, generator
from sqlglot.dialects.dialect import Dialect
from sqlglot.dialects.dialect import rename_func, Dialect
class Druid(Dialect):
@ -12,3 +12,9 @@ class Druid(Dialect):
exp.DataType.Type.TEXT: "STRING",
exp.DataType.Type.UUID: "STRING",
}
TRANSFORMS = {
**generator.Generator.TRANSFORMS,
exp.CurrentTimestamp: lambda *_: "CURRENT_TIMESTAMP",
exp.Mod: rename_func("MOD"),
}

2
sqlglot/dialects/duckdb.py
View file

@ -320,7 +320,6 @@ class DuckDB(Dialect):
"BITSTRING": TokenType.BIT,
"BPCHAR": TokenType.TEXT,
"CHAR": TokenType.TEXT,
"CHARACTER VARYING": TokenType.TEXT,
"DATETIME": TokenType.TIMESTAMPNTZ,
"DETACH": TokenType.DETACH,
"EXCLUDE": TokenType.EXCEPT,
@ -614,6 +613,7 @@ class DuckDB(Dialect):
MULTI_ARG_DISTINCT = False
CAN_IMPLEMENT_ARRAY_ANY = True
SUPPORTS_TO_NUMBER = False
SUPPORTS_WINDOW_EXCLUDE = True
COPY_HAS_INTO_KEYWORD = False
STAR_EXCEPT = "EXCLUDE"
PAD_FILL_PATTERN_IS_REQUIRED = True

20
sqlglot/dialects/hive.py
View file

@ -1,7 +1,9 @@
from __future__ import annotations
import typing as t
from copy import deepcopy
from functools import partial
from collections import defaultdict
from sqlglot import exp, generator, parser, tokens, transforms
from sqlglot.dialects.dialect import (
@ -43,6 +45,7 @@ from sqlglot.transforms import (
from sqlglot.helper import seq_get
from sqlglot.tokens import TokenType
from sqlglot.generator import unsupported_args
from sqlglot.optimizer.annotate_types import TypeAnnotator
# (FuncType, Multiplier)
DATE_DELTA_INTERVAL = {
@ -209,6 +212,23 @@ class Hive(Dialect):
# https://spark.apache.org/docs/latest/sql-ref-identifier.html#description
NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE
ANNOTATORS = {
**Dialect.ANNOTATORS,
exp.If: lambda self, e: self._annotate_by_args(e, "true", "false", promote=True),
exp.Coalesce: lambda self, e: self._annotate_by_args(
e, "this", "expressions", promote=True
),
}
# Support only the non-ANSI mode (default for Hive, Spark2, Spark)
COERCES_TO = defaultdict(set, deepcopy(TypeAnnotator.COERCES_TO))
for target_type in {
*exp.DataType.NUMERIC_TYPES,
*exp.DataType.TEMPORAL_TYPES,
exp.DataType.Type.INTERVAL,
}:
COERCES_TO[target_type] |= exp.DataType.TEXT_TYPES
TIME_MAPPING = {
"y": "%Y",
"Y": "%Y",

16
sqlglot/dialects/oracle.py
View file

@ -31,6 +31,13 @@ def _trim_sql(self: Oracle.Generator, expression: exp.Trim) -> str:
return trim_sql(self, expression)
def _build_to_timestamp(args: t.List) -> exp.StrToTime | exp.Anonymous:
if len(args) == 1:
return exp.Anonymous(this="TO_TIMESTAMP", expressions=args)
return build_formatted_time(exp.StrToTime, "oracle")(args)
class Oracle(Dialect):
ALIAS_POST_TABLESAMPLE = True
LOCKING_READS_SUPPORTED = True
@ -103,10 +110,11 @@ class Oracle(Dialect):
FUNCTIONS = {
**parser.Parser.FUNCTIONS,
"CONVERT": exp.ConvertToCharset.from_arg_list,
"NVL": lambda args: build_coalesce(args, is_nvl=True),
"SQUARE": lambda args: exp.Pow(this=seq_get(args, 0), expression=exp.Literal.number(2)),
"TO_CHAR": build_timetostr_or_tochar,
"TO_TIMESTAMP": build_formatted_time(exp.StrToTime, "oracle"),
"TO_TIMESTAMP": _build_to_timestamp,
"TO_DATE": build_formatted_time(exp.StrToDate, "oracle"),
"TRUNC": lambda args: exp.DateTrunc(
unit=seq_get(args, 1) or exp.Literal.string("DD"),
@ -118,6 +126,7 @@ class Oracle(Dialect):
NO_PAREN_FUNCTION_PARSERS = {
**parser.Parser.NO_PAREN_FUNCTION_PARSERS,
"NEXT": lambda self: self._parse_next_value_for(),
"PRIOR": lambda self: self.expression(exp.Prior, this=self._parse_bitwise()),
"SYSDATE": lambda self: self.expression(exp.CurrentTimestamp, sysdate=True),
}
@ -134,6 +143,7 @@ class Oracle(Dialect):
),
"JSON_EXISTS": lambda self: self._parse_json_exists(),
}
FUNCTION_PARSERS.pop("CONVERT")
PROPERTY_PARSERS = {
**parser.Parser.PROPERTY_PARSERS,
@ -240,6 +250,9 @@ class Oracle(Dialect):
return self.expression(exp.Into, bulk_collect=bulk_collect, expressions=expressions)
def _parse_connect_with_prior(self):
return self._parse_assignment()
class Generator(generator.Generator):
LOCKING_READS_SUPPORTED = True
JOIN_HINTS = False
@ -251,6 +264,7 @@ class Oracle(Dialect):
LAST_DAY_SUPPORTS_DATE_PART = False
SUPPORTS_SELECT_INTO = True
TZ_TO_WITH_TIME_ZONE = True
SUPPORTS_WINDOW_EXCLUDE = True
QUERY_HINT_SEP = " "
TYPE_MAPPING = {

2
sqlglot/dialects/postgres.py
View file

@ -321,7 +321,6 @@ class Postgres(Dialect):
"BEGIN": TokenType.COMMAND,
"BEGIN TRANSACTION": TokenType.BEGIN,
"BIGSERIAL": TokenType.BIGSERIAL,
"CHARACTER VARYING": TokenType.VARCHAR,
"CONSTRAINT TRIGGER": TokenType.COMMAND,
"CSTRING": TokenType.PSEUDO_TYPE,
"DECLARE": TokenType.COMMAND,
@ -519,6 +518,7 @@ class Postgres(Dialect):
LIKE_PROPERTY_INSIDE_SCHEMA = True
MULTI_ARG_DISTINCT = False
CAN_IMPLEMENT_ARRAY_ANY = True
SUPPORTS_WINDOW_EXCLUDE = True
COPY_HAS_INTO_KEYWORD = False
ARRAY_CONCAT_IS_VAR_LEN = False
SUPPORTS_MEDIAN = False

5
sqlglot/dialects/presto.py
View file

@ -220,7 +220,7 @@ def _explode_to_unnest_sql(self: Presto.Generator, expression: exp.Lateral) -> s
return explode_to_unnest_sql(self, expression)
def _amend_exploded_column_table(expression: exp.Expression) -> exp.Expression:
def amend_exploded_column_table(expression: exp.Expression) -> exp.Expression:
# We check for expression.type because the columns can be amended only if types were inferred
if isinstance(expression, exp.Select) and expression.type:
for lateral in expression.args.get("laterals") or []:
@ -484,11 +484,12 @@ class Presto(Dialect):
exp.SchemaCommentProperty: lambda self, e: self.naked_property(e),
exp.Select: transforms.preprocess(
[
transforms.eliminate_window_clause,
transforms.eliminate_qualify,
transforms.eliminate_distinct_on,
transforms.explode_projection_to_unnest(1),
transforms.eliminate_semi_and_anti_joins,
_amend_exploded_column_table,
amend_exploded_column_table,
]
),
exp.SortArray: _no_sort_array,

1
sqlglot/dialects/redshift.py
View file

@ -197,6 +197,7 @@ class Redshift(Postgres):
exp.Hex: lambda self, e: self.func("UPPER", self.func("TO_HEX", self.sql(e, "this"))),
exp.Select: transforms.preprocess(
[
transforms.eliminate_window_clause,
transforms.eliminate_distinct_on,
transforms.eliminate_semi_and_anti_joins,
transforms.unqualify_unnest,

4
sqlglot/dialects/snowflake.py
View file

@ -949,8 +949,6 @@ class Snowflake(Dialect):
**tokens.Tokenizer.KEYWORDS,
"FILE://": TokenType.URI_START,
"BYTEINT": TokenType.INT,
"CHAR VARYING": TokenType.VARCHAR,
"CHARACTER VARYING": TokenType.VARCHAR,
"EXCLUDE": TokenType.EXCEPT,
"FILE FORMAT": TokenType.FILE_FORMAT,
"GET": TokenType.GET,
@ -1080,6 +1078,7 @@ class Snowflake(Dialect):
exp.Rand: rename_func("RANDOM"),
exp.Select: transforms.preprocess(
[
transforms.eliminate_window_clause,
transforms.eliminate_distinct_on,
transforms.explode_projection_to_unnest(),
transforms.eliminate_semi_and_anti_joins,
@ -1132,6 +1131,7 @@ class Snowflake(Dialect):
**generator.Generator.TYPE_MAPPING,
exp.DataType.Type.NESTED: "OBJECT",
exp.DataType.Type.STRUCT: "OBJECT",
exp.DataType.Type.BIGDECIMAL: "DOUBLE",
}
TOKEN_MAPPING = {

1
sqlglot/dialects/sqlite.py
View file

@ -124,6 +124,7 @@ class SQLite(Dialect):
SUPPORTS_CREATE_TABLE_LIKE = False
SUPPORTS_TABLE_ALIAS_COLUMNS = False
SUPPORTS_TO_NUMBER = False
SUPPORTS_WINDOW_EXCLUDE = True
EXCEPT_INTERSECT_SUPPORT_ALL_CLAUSE = False
SUPPORTS_MEDIAN = False
JSON_KEY_VALUE_PAIR_SEP = ","

1
sqlglot/dialects/starrocks.py
View file

@ -132,6 +132,7 @@ class StarRocks(MySQL):
TRANSFORMS = {
**MySQL.Generator.TRANSFORMS,
exp.Array: inline_array_sql,
exp.ArrayToString: rename_func("ARRAY_JOIN"),
exp.ApproxDistinct: approx_count_distinct_sql,
exp.DateDiff: lambda self, e: self.func(
"DATE_DIFF", unit_to_str(e), e.this, e.expression

22
sqlglot/dialects/teradata.py
View file

@ -162,7 +162,7 @@ class Teradata(Dialect):
# https://docs.teradata.com/r/SQL-Functions-Operators-Expressions-and-Predicates/June-2017/Data-Type-Conversions/TRYCAST
"TRYCAST": parser.Parser.FUNCTION_PARSERS["TRY_CAST"],
"RANGE_N": lambda self: self._parse_rangen(),
"TRANSLATE": lambda self: self._parse_translate(self.STRICT_CAST),
"TRANSLATE": lambda self: self._parse_translate(),
}
FUNCTIONS = {
@ -175,19 +175,17 @@ class Teradata(Dialect):
TokenType.DSTAR: exp.Pow,
}
def _parse_translate(self, strict: bool) -> exp.Expression:
def _parse_translate(self) -> exp.TranslateCharacters:
this = self._parse_assignment()
self._match(TokenType.USING)
self._match_texts(self.CHARSET_TRANSLATORS)
if not self._match(TokenType.USING):
self.raise_error("Expected USING in TRANSLATE")
if self._match_texts(self.CHARSET_TRANSLATORS):
charset_split = self._prev.text.split("_TO_")
to = self.expression(exp.CharacterSet, this=charset_split[1])
else:
self.raise_error("Expected a character set translator after USING in TRANSLATE")
return self.expression(exp.Cast if strict else exp.TryCast, this=this, to=to)
return self.expression(
exp.TranslateCharacters,
this=this,
expression=self._prev.text.upper(),
with_error=self._match_text_seq("WITH", "ERROR"),
)
# FROM before SET in Teradata UPDATE syntax
# https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/Teradata-VantageTM-SQL-Data-Manipulation-Language-17.20/Statement-Syntax/UPDATE/UPDATE-Syntax-Basic-Form-FROM-Clause

13
sqlglot/dialects/trino.py
View file

@ -1,13 +1,13 @@
from __future__ import annotations
from sqlglot import exp, parser
from sqlglot import exp, parser, transforms
from sqlglot.dialects.dialect import (
merge_without_target_sql,
trim_sql,
timestrtotime_sql,
groupconcat_sql,
)
from sqlglot.dialects.presto import Presto
from sqlglot.dialects.presto import amend_exploded_column_table, Presto
from sqlglot.tokens import TokenType
import typing as t
@ -75,6 +75,15 @@ class Trino(Presto):
exp.GroupConcat: lambda self, e: groupconcat_sql(self, e, on_overflow=True),
exp.LocationProperty: lambda self, e: self.property_sql(e),
exp.Merge: merge_without_target_sql,
exp.Select: transforms.preprocess(
[
transforms.eliminate_qualify,
transforms.eliminate_distinct_on,
transforms.explode_projection_to_unnest(1),
transforms.eliminate_semi_and_anti_joins,
amend_exploded_column_table,
]
),
exp.TimeStrToTime: lambda self, e: timestrtotime_sql(self, e, include_precision=True),
exp.Trim: trim_sql,
}

15
sqlglot/dialects/tsql.py
View file

@ -10,16 +10,16 @@ from sqlglot.dialects.dialect import (
Dialect,
NormalizationStrategy,
any_value_to_max_sql,
build_date_delta,
date_delta_sql,
datestrtodate_sql,
generatedasidentitycolumnconstraint_sql,
max_or_greatest,
min_or_least,
build_date_delta,
rename_func,
strposition_sql,
trim_sql,
timestrtotime_sql,
trim_sql,
)
from sqlglot.helper import seq_get
from sqlglot.parser import build_coalesce
@ -890,6 +890,17 @@ class TSQL(Dialect):
return self.expression(exp.DeclareItem, this=var, kind=data_type, default=value)
def _parse_alter_table_alter(self) -> t.Optional[exp.Expression]:
expression = super()._parse_alter_table_alter()
if expression is not None:
collation = expression.args.get("collate")
if isinstance(collation, exp.Column) and isinstance(collation.this, exp.Identifier):
identifier = collation.this
collation.set("this", exp.Var(this=identifier.name))
return expression
class Generator(generator.Generator):
LIMIT_IS_TOP = True
QUERY_HINTS = False

3
sqlglot/executor/table.py
View file

@ -32,6 +32,9 @@ class Table:
def pop(self):
self.rows.pop()
def to_pylist(self):
return [dict(zip(self.columns, row)) for row in self.rows]
@property
def width(self):
return len(self.columns)

12
sqlglot/expressions.py
View file

@ -2497,6 +2497,7 @@ class Join(Expression):
"hint": False,
"match_condition": False, # Snowflake
"expressions": False,
"pivots": False,
}
@property
@ -4423,6 +4424,7 @@ class WindowSpec(Expression):
"start_side": False,
"end": False,
"end_side": False,
"exclude": False,
}
@ -5494,6 +5496,11 @@ class Convert(Func):
arg_types = {"this": True, "expression": True, "style": False}
# https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/CONVERT.html
class ConvertToCharset(Func):
arg_types = {"this": True, "dest": True, "source": False}
class ConvertTimezone(Func):
arg_types = {"source_tz": False, "target_tz": True, "timestamp": True}
@ -5698,6 +5705,11 @@ class CastToStrType(Func):
arg_types = {"this": True, "to": True}
# https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/SQL-Functions-Expressions-and-Predicates/String-Operators-and-Functions/TRANSLATE/TRANSLATE-Function-Syntax
class TranslateCharacters(Expression):
arg_types = {"this": True, "expression": True, "with_error": False}
class Collate(Binary, Func):
pass

29
sqlglot/generator.py
View file

@ -131,6 +131,9 @@ class Generator(metaclass=_Generator):
exp.CollateColumnConstraint: lambda self, e: f"COLLATE {self.sql(e, 'this')}",
exp.CommentColumnConstraint: lambda self, e: f"COMMENT {self.sql(e, 'this')}",
exp.ConnectByRoot: lambda self, e: f"CONNECT_BY_ROOT {self.sql(e, 'this')}",
exp.ConvertToCharset: lambda self, e: self.func(
"CONVERT", e.this, e.args["dest"], e.args.get("source")
),
exp.CopyGrantsProperty: lambda *_: "COPY GRANTS",
exp.CredentialsProperty: lambda self,
e: f"CREDENTIALS=({self.expressions(e, 'expressions', sep=' ')})",
@ -404,6 +407,9 @@ class Generator(metaclass=_Generator):
# Whether the function TO_NUMBER is supported
SUPPORTS_TO_NUMBER = True
# Whether EXCLUDE in window specification is supported
SUPPORTS_WINDOW_EXCLUDE = False
# Whether or not set op modifiers apply to the outer set op or select.
# SELECT * FROM x UNION SELECT * FROM y LIMIT 1
# True means limit 1 happens after the set op, False means it it happens on y.
@ -2290,7 +2296,8 @@ class Generator(metaclass=_Generator):
if op_sql != "STRAIGHT_JOIN":
op_sql = f"{op_sql} JOIN" if op_sql else "JOIN"
return f"{self.seg(op_sql)} {this_sql}{match_cond}{on_sql}"
pivots = self.expressions(expression, key="pivots", sep="", flat=True)
return f"{self.seg(op_sql)} {this_sql}{match_cond}{on_sql}{pivots}"
def lambda_sql(self, expression: exp.Lambda, arrow_sep: str = "->") -> str:
args = self.expressions(expression, flat=True)
@ -2806,7 +2813,17 @@ class Generator(metaclass=_Generator):
csv(self.sql(expression, "end"), self.sql(expression, "end_side"), sep=" ")
or "CURRENT ROW"
)
return f"{kind} BETWEEN {start} AND {end}"
window_spec = f"{kind} BETWEEN {start} AND {end}"
exclude = self.sql(expression, "exclude")
if exclude:
if self.SUPPORTS_WINDOW_EXCLUDE:
window_spec += f" EXCLUDE {exclude}"
else:
self.unsupported("EXCLUDE clause is not supported in the WINDOW clause")
return window_spec
def withingroup_sql(self, expression: exp.WithinGroup) -> str:
this = self.sql(expression, "this")
@ -3824,7 +3841,7 @@ class Generator(metaclass=_Generator):
if isinstance(then_expression.args.get("expressions"), exp.Star):
then = f"UPDATE {self.sql(then_expression, 'expressions')}"
else:
then = f"UPDATE SET {self.expressions(then_expression, flat=True)}"
then = f"UPDATE SET{self.sep()}{self.expressions(then_expression)}"
else:
then = self.sql(then_expression)
return f"WHEN {matched}{source}{condition} THEN {then}"
@ -4925,3 +4942,9 @@ class Generator(metaclass=_Generator):
return f"PUT {this} {target}{props_sql}"
else:
return f"GET {target} {this}{props_sql}"
def translatecharacters_sql(self, expression: exp.TranslateCharacters):
this = self.sql(expression, "this")
expr = self.sql(expression, "expression")
with_error = " WITH ERROR" if expression.args.get("with_error") else ""
return f"TRANSLATE({this} USING {expr}{with_error})"

17
sqlglot/optimizer/annotate_types.py
View file

@ -32,7 +32,7 @@ def annotate_types(
schema: t.Optional[t.Dict | Schema] = None,
annotators: t.Optional[AnnotatorsType] = None,
coerces_to: t.Optional[t.Dict[exp.DataType.Type, t.Set[exp.DataType.Type]]] = None,
dialect: t.Optional[DialectType] = None,
dialect: DialectType = None,
) -> E:
"""
Infers the types of an expression, annotating its AST accordingly.
@ -55,9 +55,9 @@ def annotate_types(
The expression annotated with types.
"""
schema = ensure_schema(schema)
schema = ensure_schema(schema, dialect=dialect)
return TypeAnnotator(schema, annotators, coerces_to, dialect=dialect).annotate(expression)
return TypeAnnotator(schema, annotators, coerces_to).annotate(expression)
def _coerce_date_literal(l: exp.Expression, unit: t.Optional[exp.Expression]) -> exp.DataType.Type:
@ -182,11 +182,12 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
annotators: t.Optional[AnnotatorsType] = None,
coerces_to: t.Optional[t.Dict[exp.DataType.Type, t.Set[exp.DataType.Type]]] = None,
binary_coercions: t.Optional[BinaryCoercions] = None,
dialect: t.Optional[DialectType] = None,
) -> None:
self.schema = schema
self.annotators = annotators or Dialect.get_or_raise(dialect).ANNOTATORS
self.coerces_to = coerces_to or self.COERCES_TO
self.annotators = annotators or Dialect.get_or_raise(schema.dialect).ANNOTATORS
self.coerces_to = (
coerces_to or Dialect.get_or_raise(schema.dialect).COERCES_TO or self.COERCES_TO
)
self.binary_coercions = binary_coercions or self.BINARY_COERCIONS
# Caches the ids of annotated sub-Expressions, to ensure we only visit them once
@ -311,7 +312,9 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
return expression
def _maybe_coerce(
self, type1: exp.DataType | exp.DataType.Type, type2: exp.DataType | exp.DataType.Type
self,
type1: exp.DataType | exp.DataType.Type,
type2: exp.DataType | exp.DataType.Type,
) -> exp.DataType | exp.DataType.Type:
"""
Returns type2 if type1 can be coerced into it, otherwise type1.

19
sqlglot/optimizer/isolate_table_selects.py
View file

@ -1,17 +1,32 @@
from __future__ import annotations
import typing as t
from sqlglot import alias, exp
from sqlglot.errors import OptimizeError
from sqlglot.optimizer.scope import traverse_scope
from sqlglot.schema import ensure_schema
if t.TYPE_CHECKING:
from sqlglot._typing import E
from sqlglot.schema import Schema
from sqlglot.dialects.dialect import DialectType
def isolate_table_selects(expression, schema=None):
schema = ensure_schema(schema)
def isolate_table_selects(
expression: E,
schema: t.Optional[t.Dict | Schema] = None,
dialect: DialectType = None,
) -> E:
schema = ensure_schema(schema, dialect=dialect)
for scope in traverse_scope(expression):
if len(scope.selected_sources) == 1:
continue
for _, source in scope.selected_sources.values():
assert source.parent
if (
not isinstance(source, exp.Table)
or not schema.column_names(source)

4
sqlglot/optimizer/merge_subqueries.py
View file

@ -92,9 +92,9 @@ def merge_ctes(expression: E, leave_tables_isolated: bool = False) -> E:
_rename_inner_sources(outer_scope, inner_scope, alias)
_merge_from(outer_scope, inner_scope, table, alias)
_merge_expressions(outer_scope, inner_scope, alias)
_merge_order(outer_scope, inner_scope)
_merge_joins(outer_scope, inner_scope, from_or_join)
_merge_where(outer_scope, inner_scope, from_or_join)
_merge_order(outer_scope, inner_scope)
_merge_hints(outer_scope, inner_scope)
_pop_cte(inner_scope)
outer_scope.clear_cache()
@ -111,9 +111,9 @@ def merge_derived_tables(expression: E, leave_tables_isolated: bool = False) ->
_rename_inner_sources(outer_scope, inner_scope, alias)
_merge_from(outer_scope, inner_scope, subquery, alias)
_merge_expressions(outer_scope, inner_scope, alias)
_merge_order(outer_scope, inner_scope)
_merge_joins(outer_scope, inner_scope, from_or_join)
_merge_where(outer_scope, inner_scope, from_or_join)
_merge_order(outer_scope, inner_scope)
_merge_hints(outer_scope, inner_scope)
outer_scope.clear_cache()

25
sqlglot/optimizer/pushdown_projections.py
View file

@ -1,3 +1,6 @@
from __future__ import annotations
import typing as t
from collections import defaultdict
from sqlglot import alias, exp
@ -7,6 +10,11 @@ from sqlglot.schema import ensure_schema
from sqlglot.errors import OptimizeError
from sqlglot.helper import seq_get
if t.TYPE_CHECKING:
from sqlglot._typing import E
from sqlglot.schema import Schema
from sqlglot.dialects.dialect import DialectType
# Sentinel value that means an outer query selecting ALL columns
SELECT_ALL = object()
@ -16,7 +24,12 @@ def default_selection(is_agg: bool) -> exp.Alias:
return alias(exp.Max(this=exp.Literal.number(1)) if is_agg else "1", "_")
def pushdown_projections(expression, schema=None, remove_unused_selections=True):
def pushdown_projections(
expression: E,
schema: t.Optional[t.Dict | Schema] = None,
remove_unused_selections: bool = True,
dialect: DialectType = None,
) -> E:
"""
Rewrite sqlglot AST to remove unused columns projections.
@ -34,9 +47,9 @@ def pushdown_projections(expression, schema=None, remove_unused_selections=True)
sqlglot.Expression: optimized expression
"""
# Map of Scope to all columns being selected by outer queries.
schema = ensure_schema(schema)
source_column_alias_count = {}
referenced_columns = defaultdict(set)
schema = ensure_schema(schema, dialect=dialect)
source_column_alias_count: t.Dict[exp.Expression | Scope, int] = {}
referenced_columns: t.DefaultDict[Scope, t.Set[str | object]] = defaultdict(set)
# We build the scope tree (which is traversed in DFS postorder), then iterate
# over the result in reverse order. This should ensure that the set of selected
@ -69,12 +82,12 @@ def pushdown_projections(expression, schema=None, remove_unused_selections=True)
if scope.expression.args.get("by_name"):
referenced_columns[right] = referenced_columns[left]
else:
referenced_columns[right] = [
referenced_columns[right] = {
right.expression.selects[i].alias_or_name
for i, select in enumerate(left.expression.selects)
if SELECT_ALL in parent_selections
or select.alias_or_name in parent_selections
]
}
if isinstance(scope.expression, exp.Select):
if remove_unused_selections:

3
sqlglot/optimizer/qualify_columns.py
View file

@ -23,6 +23,7 @@ def qualify_columns(
expand_stars: bool = True,
infer_schema: t.Optional[bool] = None,
allow_partial_qualification: bool = False,
dialect: DialectType = None,
) -> exp.Expression:
"""
Rewrite sqlglot AST to have fully qualified columns.
@ -50,7 +51,7 @@ def qualify_columns(
Notes:
- Currently only handles a single PIVOT or UNPIVOT operator
"""
schema = ensure_schema(schema)
schema = ensure_schema(schema, dialect=dialect)
annotator = TypeAnnotator(schema)
infer_schema = schema.empty if infer_schema is None else infer_schema
dialect = Dialect.get_or_raise(schema.dialect)

36
sqlglot/parser.py
View file

@ -1345,6 +1345,12 @@ class Parser(metaclass=_Parser):
**dict.fromkeys(("DEFERRABLE", "NORELY", "RELY"), tuple()),
}
WINDOW_EXCLUDE_OPTIONS: OPTIONS_TYPE = {
"NO": ("OTHERS",),
"CURRENT": ("ROW",),
**dict.fromkeys(("GROUP", "TIES"), tuple()),
}
INSERT_ALTERNATIVES = {"ABORT", "FAIL", "IGNORE", "REPLACE", "ROLLBACK"}
CLONE_KEYWORDS = {"CLONE", "COPY"}
@ -2235,7 +2241,7 @@ class Parser(metaclass=_Parser):
)
def _parse_security(self) -> t.Optional[exp.SecurityProperty]:
if self._match_texts(("DEFINER", "INVOKER")):
if self._match_texts(("NONE", "DEFINER", "INVOKER")):
security_specifier = self._prev.text.upper()
return self.expression(exp.SecurityProperty, this=security_specifier)
return None
@ -3732,6 +3738,8 @@ class Parser(metaclass=_Parser):
kwargs["this"].set("joins", joins if joins else None)
kwargs["pivots"] = self._parse_pivots()
comments = [c for token in (method, side, kind) if token for c in token.comments]
return self.expression(exp.Join, comments=comments, **kwargs)
@ -4455,6 +4463,14 @@ class Parser(metaclass=_Parser):
return None
return self.expression(exp.Qualify, this=self._parse_assignment())
def _parse_connect_with_prior(self) -> t.Optional[exp.Expression]:
self.NO_PAREN_FUNCTION_PARSERS["PRIOR"] = lambda self: self.expression(
exp.Prior, this=self._parse_bitwise()
)
connect = self._parse_assignment()
self.NO_PAREN_FUNCTION_PARSERS.pop("PRIOR")
return connect
def _parse_connect(self, skip_start_token: bool = False) -> t.Optional[exp.Connect]:
if skip_start_token:
start = None
@ -4465,11 +4481,7 @@ class Parser(metaclass=_Parser):
self._match(TokenType.CONNECT_BY)
nocycle = self._match_text_seq("NOCYCLE")
self.NO_PAREN_FUNCTION_PARSERS["PRIOR"] = lambda self: self.expression(
exp.Prior, this=self._parse_bitwise()
)
connect = self._parse_assignment()
self.NO_PAREN_FUNCTION_PARSERS.pop("PRIOR")
connect = self._parse_connect_with_prior()
if not start and self._match(TokenType.START_WITH):
start = self._parse_assignment()
@ -6088,7 +6100,11 @@ class Parser(metaclass=_Parser):
return self.expression(exp.Reference, this=this, expressions=expressions, options=options)
def _parse_foreign_key(self) -> exp.ForeignKey:
expressions = self._parse_wrapped_id_vars()
expressions = (
self._parse_wrapped_id_vars()
if not self._match(TokenType.REFERENCES, advance=False)
else None
)
reference = self._parse_references()
on_options = {}
@ -6887,6 +6903,11 @@ class Parser(metaclass=_Parser):
start = self._parse_window_spec()
self._match(TokenType.AND)
end = self._parse_window_spec()
exclude = (
self._parse_var_from_options(self.WINDOW_EXCLUDE_OPTIONS)
if self._match_text_seq("EXCLUDE")
else None
)
spec = self.expression(
exp.WindowSpec,
@ -6895,6 +6916,7 @@ class Parser(metaclass=_Parser):
start_side=start["side"],
end=end["value"],
end_side=end["side"],
exclude=exclude,
)
else:
spec = None

4
sqlglot/tokens.py
View file

@ -907,6 +907,8 @@ class Tokenizer(metaclass=_Tokenizer):
"JSONB": TokenType.JSONB,
"CHAR": TokenType.CHAR,
"CHARACTER": TokenType.CHAR,
"CHAR VARYING": TokenType.VARCHAR,
"CHARACTER VARYING": TokenType.VARCHAR,
"NCHAR": TokenType.NCHAR,
"VARCHAR": TokenType.VARCHAR,
"VARCHAR2": TokenType.VARCHAR,
@ -1109,7 +1111,7 @@ class Tokenizer(metaclass=_Tokenizer):
if self.WHITE_SPACE.get(self._char) is TokenType.BREAK:
# Ensures we don't count an extra line if we get a \r\n line break sequence
if not (self._char == "\r" and self._peek == "\n"):
self._col = 1
self._col = i
self._line += 1
else:
self._col += i

46
sqlglot/transforms.py
View file

@ -309,10 +309,9 @@ def unqualify_unnest(expression: exp.Expression) -> exp.Expression:
}
if unnest_aliases:
for column in expression.find_all(exp.Column):
if column.table in unnest_aliases:
column.set("table", None)
elif column.db in unnest_aliases:
column.set("db", None)
leftmost_part = column.parts[0]
if leftmost_part.arg_key != "this" and leftmost_part.this in unnest_aliases:
leftmost_part.pop()
return expression
@ -966,16 +965,47 @@ def any_to_exists(expression: exp.Expression) -> exp.Expression:
transformation
"""
if isinstance(expression, exp.Select):
for any in expression.find_all(exp.Any):
this = any.this
for any_expr in expression.find_all(exp.Any):
this = any_expr.this
if isinstance(this, exp.Query):
continue
binop = any.parent
binop = any_expr.parent
if isinstance(binop, exp.Binary):
lambda_arg = exp.to_identifier("x")
any.replace(lambda_arg)
any_expr.replace(lambda_arg)
lambda_expr = exp.Lambda(this=binop.copy(), expressions=[lambda_arg])
binop.replace(exp.Exists(this=this.unnest(), expression=lambda_expr))
return expression
def eliminate_window_clause(expression: exp.Expression) -> exp.Expression:
"""Eliminates the `WINDOW` query clause by inling each named window."""
if isinstance(expression, exp.Select) and expression.args.get("windows"):
from sqlglot.optimizer.scope import find_all_in_scope
windows = expression.args["windows"]
expression.set("windows", None)
window_expression: t.Dict[str, exp.Expression] = {}
def _inline_inherited_window(window: exp.Expression) -> None:
inherited_window = window_expression.get(window.alias.lower())
if not inherited_window:
return
window.set("alias", None)
for key in ("partition_by", "order", "spec"):
arg = inherited_window.args.get(key)
if arg:
window.set(key, arg.copy())
for window in windows:
_inline_inherited_window(window)
window_expression[window.name.lower()] = window
for window in find_all_in_scope(expression, exp.Window):
_inline_inherited_window(window)
return expression

2
sqlglotrs/Cargo.lock generated
View file

@ -503,7 +503,7 @@ dependencies = [
[[package]]
name = "sqlglotrs"
version = "0.4.0"
version = "0.4.2"
dependencies = [
"criterion",
"pyo3",

2
sqlglotrs/Cargo.toml
View file

@ -1,6 +1,6 @@
[package]
name = "sqlglotrs"
version = "0.4.0"
version = "0.4.2"
edition = "2021"
license = "MIT"

2
sqlglotrs/src/tokenizer.rs
View file

@ -173,7 +173,7 @@ impl<'a> TokenizerState<'a> {
if Some(&self.token_types.break_) == self.settings.white_space.get(&self.current_char) {
// Ensures we don't count an extra line if we get a \r\n line break sequence.
if !(self.current_char == '\r' && self.peek_char == '\n') {
self.column = 1;
self.column = i as usize;
self.line += 1;
}
} else {

186
tests/dialects/test_bigquery.py
View file

@ -16,6 +16,7 @@ from sqlglot.helper import logger as helper_logger
from sqlglot.parser import logger as parser_logger
from tests.dialects.test_dialect import Validator
from sqlglot.optimizer.annotate_types import annotate_types
from sqlglot.optimizer.qualify import qualify
class TestBigQuery(Validator):
@ -23,67 +24,6 @@ class TestBigQuery(Validator):
maxDiff = None
def test_bigquery(self):
self.validate_all(
"EXTRACT(HOUR FROM DATETIME(2008, 12, 25, 15, 30, 00))",
write={
"bigquery": "EXTRACT(HOUR FROM DATETIME(2008, 12, 25, 15, 30, 00))",
"duckdb": "EXTRACT(HOUR FROM MAKE_TIMESTAMP(2008, 12, 25, 15, 30, 00))",
"snowflake": "DATE_PART(HOUR, TIMESTAMP_FROM_PARTS(2008, 12, 25, 15, 30, 00))",
},
)
self.validate_identity(
"""CREATE TEMPORARY FUNCTION FOO()
RETURNS STRING
LANGUAGE js AS
'return "Hello world!"'""",
pretty=True,
)
self.validate_identity(
"[a, a(1, 2,3,4444444444444444, tttttaoeunthaoentuhaoentuheoantu, toheuntaoheutnahoeunteoahuntaoeh), b(3, 4,5), c, d, tttttttttttttttteeeeeeeeeeeeeett, 12312312312]",
"""[
a,
a(
1,
2,
3,
4444444444444444,
tttttaoeunthaoentuhaoentuheoantu,
toheuntaoheutnahoeunteoahuntaoeh
),
b(3, 4, 5),
c,
d,
tttttttttttttttteeeeeeeeeeeeeett,
12312312312
]""",
pretty=True,
)
self.validate_all(
"SELECT STRUCT(1, 2, 3), STRUCT(), STRUCT('abc'), STRUCT(1, t.str_col), STRUCT(1 as a, 'abc' AS b), STRUCT(str_col AS abc)",
write={
"bigquery": "SELECT STRUCT(1, 2, 3), STRUCT(), STRUCT('abc'), STRUCT(1, t.str_col), STRUCT(1 AS a, 'abc' AS b), STRUCT(str_col AS abc)",
"duckdb": "SELECT {'_0': 1, '_1': 2, '_2': 3}, {}, {'_0': 'abc'}, {'_0': 1, '_1': t.str_col}, {'a': 1, 'b': 'abc'}, {'abc': str_col}",
"hive": "SELECT STRUCT(1, 2, 3), STRUCT(), STRUCT('abc'), STRUCT(1, t.str_col), STRUCT(1, 'abc'), STRUCT(str_col)",
"spark2": "SELECT STRUCT(1, 2, 3), STRUCT(), STRUCT('abc'), STRUCT(1, t.str_col), STRUCT(1 AS a, 'abc' AS b), STRUCT(str_col AS abc)",
"spark": "SELECT STRUCT(1, 2, 3), STRUCT(), STRUCT('abc'), STRUCT(1, t.str_col), STRUCT(1 AS a, 'abc' AS b), STRUCT(str_col AS abc)",
"snowflake": "SELECT OBJECT_CONSTRUCT('_0', 1, '_1', 2, '_2', 3), OBJECT_CONSTRUCT(), OBJECT_CONSTRUCT('_0', 'abc'), OBJECT_CONSTRUCT('_0', 1, '_1', t.str_col), OBJECT_CONSTRUCT('a', 1, 'b', 'abc'), OBJECT_CONSTRUCT('abc', str_col)",
# fallback to unnamed without type inference
"trino": "SELECT ROW(1, 2, 3), ROW(), ROW('abc'), ROW(1, t.str_col), CAST(ROW(1, 'abc') AS ROW(a INTEGER, b VARCHAR)), ROW(str_col)",
},
)
self.validate_all(
"PARSE_TIMESTAMP('%Y-%m-%dT%H:%M:%E6S%z', x)",
write={
"bigquery": "PARSE_TIMESTAMP('%Y-%m-%dT%H:%M:%E6S%z', x)",
"duckdb": "STRPTIME(x, '%Y-%m-%dT%H:%M:%S.%f%z')",
},
)
self.validate_identity(
"PARSE_TIMESTAMP('%Y-%m-%dT%H:%M:%E*S%z', x)",
"PARSE_TIMESTAMP('%Y-%m-%dT%H:%M:%E*S%z', x)",
)
for prefix in ("c.db.", "db.", ""):
with self.subTest(f"Parsing {prefix}INFORMATION_SCHEMA.X into a Table"):
table = self.parse_one(f"`{prefix}INFORMATION_SCHEMA.X`", into=exp.Table)
@ -115,6 +55,7 @@ LANGUAGE js AS
select_with_quoted_udf = self.validate_identity("SELECT `p.d.UdF`(data) FROM `p.d.t`")
self.assertEqual(select_with_quoted_udf.selects[0].name, "p.d.UdF")
self.validate_identity("PARSE_TIMESTAMP('%Y-%m-%dT%H:%M:%E*S%z', x)")
self.validate_identity("SELECT ARRAY_CONCAT([1])")
self.validate_identity("SELECT * FROM READ_CSV('bla.csv')")
self.validate_identity("CAST(x AS STRUCT<list ARRAY<INT64>>)")
@ -320,7 +261,80 @@ LANGUAGE js AS
"SELECT CAST(1 AS BYTEINT)",
"SELECT CAST(1 AS INT64)",
)
self.validate_identity(
"""CREATE TEMPORARY FUNCTION FOO()
RETURNS STRING
LANGUAGE js AS
'return "Hello world!"'""",
pretty=True,
)
self.validate_identity(
"[a, a(1, 2,3,4444444444444444, tttttaoeunthaoentuhaoentuheoantu, toheuntaoheutnahoeunteoahuntaoeh), b(3, 4,5), c, d, tttttttttttttttteeeeeeeeeeeeeett, 12312312312]",
"""[
a,
a(
1,
2,
3,
4444444444444444,
tttttaoeunthaoentuhaoentuheoantu,
toheuntaoheutnahoeunteoahuntaoeh
),
b(3, 4, 5),
c,
d,
tttttttttttttttteeeeeeeeeeeeeett,
12312312312
]""",
pretty=True,
)
self.validate_all(
"SELECT purchases, LAST_VALUE(item) OVER item_window AS most_popular FROM Produce WINDOW item_window AS (PARTITION BY purchases ORDER BY purchases ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING)",
write={
"bigquery": "SELECT purchases, LAST_VALUE(item) OVER item_window AS most_popular FROM Produce WINDOW item_window AS (PARTITION BY purchases ORDER BY purchases ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING)",
"clickhouse": "SELECT purchases, LAST_VALUE(item) OVER item_window AS most_popular FROM Produce WINDOW item_window AS (PARTITION BY purchases ORDER BY purchases NULLS FIRST ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING)",
"databricks": "SELECT purchases, LAST_VALUE(item) OVER item_window AS most_popular FROM Produce WINDOW item_window AS (PARTITION BY purchases ORDER BY purchases ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING)",
"duckdb": "SELECT purchases, LAST_VALUE(item) OVER item_window AS most_popular FROM Produce WINDOW item_window AS (PARTITION BY purchases ORDER BY purchases NULLS FIRST ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING)",
"mysql": "SELECT purchases, LAST_VALUE(item) OVER item_window AS most_popular FROM Produce WINDOW item_window AS (PARTITION BY purchases ORDER BY purchases ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING)",
"oracle": "SELECT purchases, LAST_VALUE(item) OVER item_window AS most_popular FROM Produce WINDOW item_window AS (PARTITION BY purchases ORDER BY purchases NULLS FIRST ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING)",
"postgres": "SELECT purchases, LAST_VALUE(item) OVER item_window AS most_popular FROM Produce WINDOW item_window AS (PARTITION BY purchases ORDER BY purchases NULLS FIRST ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING)",
"presto": "SELECT purchases, LAST_VALUE(item) OVER (PARTITION BY purchases ORDER BY purchases NULLS FIRST ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING) AS most_popular FROM Produce",
"redshift": "SELECT purchases, LAST_VALUE(item) OVER (PARTITION BY purchases ORDER BY purchases NULLS FIRST ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING) AS most_popular FROM Produce",
"snowflake": "SELECT purchases, LAST_VALUE(item) OVER (PARTITION BY purchases ORDER BY purchases NULLS FIRST ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING) AS most_popular FROM Produce",
"spark": "SELECT purchases, LAST_VALUE(item) OVER item_window AS most_popular FROM Produce WINDOW item_window AS (PARTITION BY purchases ORDER BY purchases ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING)",
"trino": "SELECT purchases, LAST_VALUE(item) OVER item_window AS most_popular FROM Produce WINDOW item_window AS (PARTITION BY purchases ORDER BY purchases NULLS FIRST ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING)",
"tsql": "SELECT purchases, LAST_VALUE(item) OVER item_window AS most_popular FROM Produce WINDOW item_window AS (PARTITION BY purchases ORDER BY purchases ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING)",
},
)
self.validate_all(
"EXTRACT(HOUR FROM DATETIME(2008, 12, 25, 15, 30, 00))",
write={
"bigquery": "EXTRACT(HOUR FROM DATETIME(2008, 12, 25, 15, 30, 00))",
"duckdb": "EXTRACT(HOUR FROM MAKE_TIMESTAMP(2008, 12, 25, 15, 30, 00))",
"snowflake": "DATE_PART(HOUR, TIMESTAMP_FROM_PARTS(2008, 12, 25, 15, 30, 00))",
},
)
self.validate_all(
"SELECT STRUCT(1, 2, 3), STRUCT(), STRUCT('abc'), STRUCT(1, t.str_col), STRUCT(1 as a, 'abc' AS b), STRUCT(str_col AS abc)",
write={
"bigquery": "SELECT STRUCT(1, 2, 3), STRUCT(), STRUCT('abc'), STRUCT(1, t.str_col), STRUCT(1 AS a, 'abc' AS b), STRUCT(str_col AS abc)",
"duckdb": "SELECT {'_0': 1, '_1': 2, '_2': 3}, {}, {'_0': 'abc'}, {'_0': 1, '_1': t.str_col}, {'a': 1, 'b': 'abc'}, {'abc': str_col}",
"hive": "SELECT STRUCT(1, 2, 3), STRUCT(), STRUCT('abc'), STRUCT(1, t.str_col), STRUCT(1, 'abc'), STRUCT(str_col)",
"spark2": "SELECT STRUCT(1, 2, 3), STRUCT(), STRUCT('abc'), STRUCT(1, t.str_col), STRUCT(1 AS a, 'abc' AS b), STRUCT(str_col AS abc)",
"spark": "SELECT STRUCT(1, 2, 3), STRUCT(), STRUCT('abc'), STRUCT(1, t.str_col), STRUCT(1 AS a, 'abc' AS b), STRUCT(str_col AS abc)",
"snowflake": "SELECT OBJECT_CONSTRUCT('_0', 1, '_1', 2, '_2', 3), OBJECT_CONSTRUCT(), OBJECT_CONSTRUCT('_0', 'abc'), OBJECT_CONSTRUCT('_0', 1, '_1', t.str_col), OBJECT_CONSTRUCT('a', 1, 'b', 'abc'), OBJECT_CONSTRUCT('abc', str_col)",
# fallback to unnamed without type inference
"trino": "SELECT ROW(1, 2, 3), ROW(), ROW('abc'), ROW(1, t.str_col), CAST(ROW(1, 'abc') AS ROW(a INTEGER, b VARCHAR)), ROW(str_col)",
},
)
self.validate_all(
"PARSE_TIMESTAMP('%Y-%m-%dT%H:%M:%E6S%z', x)",
write={
"bigquery": "PARSE_TIMESTAMP('%Y-%m-%dT%H:%M:%E6S%z', x)",
"duckdb": "STRPTIME(x, '%Y-%m-%dT%H:%M:%S.%f%z')",
},
)
self.validate_all(
"SELECT DATE_SUB(CURRENT_DATE(), INTERVAL 2 DAY)",
write={
@ -1684,6 +1698,39 @@ WHERE
"EXPORT DATA WITH CONNECTION myproject.us.myconnection OPTIONS (URI='gs://path*.csv.gz', FORMAT='CSV') AS SELECT * FROM all_rows"
)
self.validate_all(
"SELECT * FROM t1, UNNEST(`t1`) AS `col`",
read={
"duckdb": 'SELECT * FROM t1, UNNEST("t1") "t1" ("col")',
},
write={
"bigquery": "SELECT * FROM t1, UNNEST(`t1`) AS `col`",
"redshift": 'SELECT * FROM t1, "t1" AS "col"',
},
)
self.validate_all(
"SELECT * FROM t, UNNEST(`t2`.`t3`) AS `col`",
read={
"duckdb": 'SELECT * FROM t, UNNEST("t1"."t2"."t3") "t1" ("col")',
},
write={
"bigquery": "SELECT * FROM t, UNNEST(`t2`.`t3`) AS `col`",
"redshift": 'SELECT * FROM t, "t2"."t3" AS "col"',
},
)
self.validate_all(
"SELECT * FROM t1, UNNEST(`t1`.`t2`.`t3`.`t4`) AS `col`",
read={
"duckdb": 'SELECT * FROM t1, UNNEST("t1"."t2"."t3"."t4") "t3" ("col")',
},
write={
"bigquery": "SELECT * FROM t1, UNNEST(`t1`.`t2`.`t3`.`t4`) AS `col`",
"redshift": 'SELECT * FROM t1, "t1"."t2"."t3"."t4" AS "col"',
},
)
def test_errors(self):
with self.assertRaises(TokenError):
transpile("'\\'", read="bigquery")
@ -2489,3 +2536,20 @@ OPTIONS (
information_schema_sql[table_meta["start"] : table_meta["end"] + 1]
== "`region.INFORMATION_SCHEMA.COLUMNS`"
)
def test_override_normalization_strategy(self):
sql = "SELECT * FROM p.d.t"
ast = self.parse_one(sql)
qualified = qualify(ast.copy(), dialect="bigquery,normalization_strategy=uppercase")
self.assertEqual(qualified.sql("bigquery"), "SELECT * FROM `P`.`D`.`T` AS `T`")
from sqlglot.dialects import BigQuery
from sqlglot.dialects.dialect import NormalizationStrategy
try:
BigQuery.NORMALIZATION_STRATEGY = NormalizationStrategy.UPPERCASE
qualified = qualify(ast.copy(), dialect="bigquery,normalization_strategy=uppercase")
self.assertEqual(qualified.sql("bigquery"), "SELECT * FROM `P`.`D`.`T` AS `T`")
finally:
BigQuery.NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_INSENSITIVE

1
tests/dialects/test_clickhouse.py
View file

@ -110,6 +110,7 @@ class TestClickhouse(Validator):
self.validate_identity("TRUNCATE DATABASE db")
self.validate_identity("TRUNCATE DATABASE db ON CLUSTER test_cluster")
self.validate_identity("TRUNCATE DATABASE db ON CLUSTER '{cluster}'")
self.validate_identity("EXCHANGE TABLES x.a AND y.b", check_command_warning=True)
self.validate_identity(
"SELECT DATE_BIN(toDateTime('2023-01-01 14:45:00'), INTERVAL '1' MINUTE, toDateTime('2023-01-01 14:35:30'), 'UTC')",
)

6
tests/dialects/test_databricks.py
View file

@ -162,6 +162,12 @@ class TestDatabricks(Validator):
},
)
for option in ("", " (foo)", " MATCH FULL", " NOT ENFORCED"):
with self.subTest(f"Databricks foreign key REFERENCES option: {option}."):
self.validate_identity(
f"CREATE TABLE t1 (foo BIGINT NOT NULL CONSTRAINT foo_c FOREIGN KEY REFERENCES t2{option})"
)
# https://docs.databricks.com/sql/language-manual/functions/colonsign.html
def test_json(self):
self.validate_identity("SELECT c1:price, c1:price.foo, c1:price.bar[1]")

69
tests/dialects/test_dialect.py
View file

@ -326,6 +326,46 @@ class TestDialect(Validator):
"doris": "CAST(a AS VARCHAR(3))",
},
)
self.validate_all(
"CAST(a AS CHARACTER VARYING)",
write={
"bigquery": "CAST(a AS STRING)",
"drill": "CAST(a AS VARCHAR)",
"duckdb": "CAST(a AS TEXT)",
"materialize": "CAST(a AS VARCHAR)",
"mysql": "CAST(a AS CHAR)",
"hive": "CAST(a AS STRING)",
"oracle": "CAST(a AS VARCHAR2)",
"postgres": "CAST(a AS VARCHAR)",
"presto": "CAST(a AS VARCHAR)",
"redshift": "CAST(a AS VARCHAR)",
"snowflake": "CAST(a AS VARCHAR)",
"spark": "CAST(a AS STRING)",
"starrocks": "CAST(a AS VARCHAR)",
"tsql": "CAST(a AS VARCHAR)",
"doris": "CAST(a AS VARCHAR)",
},
)
self.validate_all(
"CAST(a AS CHARACTER VARYING(3))",
write={
"bigquery": "CAST(a AS STRING)",
"drill": "CAST(a AS VARCHAR(3))",
"duckdb": "CAST(a AS TEXT(3))",
"materialize": "CAST(a AS VARCHAR(3))",
"mysql": "CAST(a AS CHAR(3))",
"hive": "CAST(a AS VARCHAR(3))",
"oracle": "CAST(a AS VARCHAR2(3))",
"postgres": "CAST(a AS VARCHAR(3))",
"presto": "CAST(a AS VARCHAR(3))",
"redshift": "CAST(a AS VARCHAR(3))",
"snowflake": "CAST(a AS VARCHAR(3))",
"spark": "CAST(a AS VARCHAR(3))",
"starrocks": "CAST(a AS VARCHAR(3))",
"tsql": "CAST(a AS VARCHAR(3))",
"doris": "CAST(a AS VARCHAR(3))",
},
)
self.validate_all(
"CAST(a AS SMALLINT)",
write={
@ -2709,6 +2749,35 @@ SELECT
},
)
def test_window_exclude(self):
for option in ("CURRENT ROW", "TIES", "GROUP"):
self.validate_all(
f"SELECT SUM(X) OVER (PARTITION BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW EXCLUDE {option})",
write={
"duckdb": f"SELECT SUM(X) OVER (PARTITION BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW EXCLUDE {option})",
"postgres": f"SELECT SUM(X) OVER (PARTITION BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW EXCLUDE {option})",
"sqlite": f"SELECT SUM(X) OVER (PARTITION BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW EXCLUDE {option})",
"oracle": f"SELECT SUM(X) OVER (PARTITION BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW EXCLUDE {option})",
},
)
# EXCLUDE NO OTHERS is the default behaviour
self.validate_all(
"SELECT SUM(X) OVER (PARTITION BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW)",
read={
"duckdb": "SELECT SUM(X) OVER (PARTITION BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW EXCLUDE NO OTHERS)",
"postgres": "SELECT SUM(X) OVER (PARTITION BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW EXCLUDE NO OTHERS)",
"sqlite": "SELECT SUM(X) OVER (PARTITION BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW EXCLUDE NO OTHERS)",
"oracle": "SELECT SUM(X) OVER (PARTITION BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW EXCLUDE NO OTHERS)",
},
write={
"duckdb": "SELECT SUM(X) OVER (PARTITION BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW)",
"postgres": "SELECT SUM(X) OVER (PARTITION BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW)",
"sqlite": "SELECT SUM(X) OVER (PARTITION BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW)",
"oracle": "SELECT SUM(X) OVER (PARTITION BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW)",
},
)
def test_nested_ctes(self):
self.validate_all(
"SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",

2
tests/dialects/test_druid.py
View file

@ -6,12 +6,14 @@ class TestDruid(Validator):
dialect = "druid"
def test_druid(self):
self.validate_identity("SELECT MOD(1000, 60)")
self.validate_identity("SELECT CEIL(__time TO WEEK) FROM t")
self.validate_identity("SELECT CEIL(col) FROM t")
self.validate_identity("SELECT CEIL(price, 2) AS rounded_price FROM t")
self.validate_identity("SELECT FLOOR(__time TO WEEK) FROM t")
self.validate_identity("SELECT FLOOR(col) FROM t")
self.validate_identity("SELECT FLOOR(price, 2) AS rounded_price FROM t")
self.validate_identity("SELECT CURRENT_TIMESTAMP")
# validate across all dialects
write = {dialect.value: "FLOOR(__time TO WEEK)" for dialect in Dialects}

2
tests/dialects/test_duckdb.py
View file

@ -9,6 +9,8 @@ class TestDuckDB(Validator):
dialect = "duckdb"
def test_duckdb(self):
self.validate_identity("SELECT UUIDV7()")
self.validate_identity("SELECT TRY(LOG(0))")
self.validate_identity("x::timestamp", "CAST(x AS TIMESTAMP)")
self.validate_identity("x::timestamp without time zone", "CAST(x AS TIMESTAMP)")
self.validate_identity("x::timestamp with time zone", "CAST(x AS TIMESTAMPTZ)")

16
tests/dialects/test_oracle.py
View file

@ -1,4 +1,4 @@
from sqlglot import exp, UnsupportedError
from sqlglot import exp, UnsupportedError, ParseError, parse_one
from tests.dialects.test_dialect import Validator
@ -47,6 +47,9 @@ class TestOracle(Validator):
self.validate_identity("SELECT * FROM V$SESSION")
self.validate_identity("SELECT TO_DATE('January 15, 1989, 11:00 A.M.')")
self.validate_identity("SELECT INSTR(haystack, needle)")
self.validate_identity(
"SELECT * FROM consumer LEFT JOIN groceries ON consumer.groceries_id = consumer.id PIVOT(MAX(type_id) FOR consumer_type IN (1, 2, 3, 4))"
)
self.validate_identity(
"SELECT * FROM test UNPIVOT INCLUDE NULLS (value FOR Description IN (col AS 'PREFIX ' || CHR(38) || ' SUFFIX'))"
)
@ -320,6 +323,7 @@ class TestOracle(Validator):
},
)
self.validate_identity("CREATE OR REPLACE FORCE VIEW foo1.foo2")
self.validate_identity("TO_TIMESTAMP('foo')")
def test_join_marker(self):
self.validate_identity("SELECT e1.x, e2.x FROM e e1, e e2 WHERE e1.y (+) = e2.y")
@ -522,6 +526,8 @@ FROM JSON_TABLE(res, '$.info[*]' COLUMNS(
)) src""",
pretty=True,
)
self.validate_identity("CONVERT('foo', 'dst')")
self.validate_identity("CONVERT('foo', 'dst', 'src')")
def test_connect_by(self):
start = "START WITH last_name = 'King'"
@ -702,3 +708,11 @@ CONNECT BY PRIOR employee_id = manager_id AND LEVEL <= 4"""
self.validate_identity(
"ANALYZE TABLE tbl VALIDATE STRUCTURE CASCADE COMPLETE OFFLINE INTO db.tbl"
)
def test_prior(self):
self.validate_identity(
"SELECT id, PRIOR name AS parent_name, name FROM tree CONNECT BY NOCYCLE PRIOR id = parent_id"
)
with self.assertRaises(ParseError):
parse_one("PRIOR as foo", read="oracle")

16
tests/dialects/test_snowflake.py
View file

@ -1055,6 +1055,22 @@ class TestSnowflake(Validator):
},
)
with self.assertRaises(ParseError):
parse_one(
"SELECT id, PRIOR name AS parent_name, name FROM tree CONNECT BY NOCYCLE PRIOR id = parent_id",
dialect="snowflake",
)
self.validate_all(
"SELECT CAST(1 AS DOUBLE), CAST(1 AS DOUBLE)",
read={
"bigquery": "SELECT CAST(1 AS BIGDECIMAL), CAST(1 AS BIGNUMERIC)",
},
write={
"snowflake": "SELECT CAST(1 AS DOUBLE), CAST(1 AS DOUBLE)",
},
)
def test_null_treatment(self):
self.validate_all(
r"SELECT FIRST_VALUE(TABLE1.COLUMN1) OVER (PARTITION BY RANDOM_COLUMN1, RANDOM_COLUMN2 ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS MY_ALIAS FROM TABLE1",

5
tests/dialects/test_starrocks.py
View file

@ -6,6 +6,8 @@ class TestStarrocks(Validator):
dialect = "starrocks"
def test_starrocks(self):
self.validate_identity("SELECT ARRAY_JOIN([1, 3, 5, NULL], '_', 'NULL')")
self.validate_identity("SELECT ARRAY_JOIN([1, 3, 5, NULL], '_')")
self.validate_identity("ALTER TABLE a SWAP WITH b")
def test_ddl(self):
@ -36,6 +38,9 @@ class TestStarrocks(Validator):
self.validate_identity(
"CREATE TABLE foo (col1 LARGEINT) DISTRIBUTED BY HASH (col1) BUCKETS 1"
)
self.validate_identity(
"CREATE VIEW foo (foo_col1) SECURITY NONE AS SELECT bar_col1 FROM bar"
)
def test_identity(self):
self.validate_identity("SELECT CAST(`a`.`b` AS INT) FROM foo")

9
tests/dialects/test_teradata.py
View file

@ -41,13 +41,8 @@ class TestTeradata(Validator):
).assert_is(exp.Command)
def test_translate(self):
self.validate_all(
"TRANSLATE(x USING LATIN_TO_UNICODE)",
write={
"teradata": "CAST(x AS CHAR CHARACTER SET UNICODE)",
},
)
self.validate_identity("CAST(x AS CHAR CHARACTER SET UNICODE)")
self.validate_identity("TRANSLATE(x USING LATIN_TO_UNICODE)")
self.validate_identity("TRANSLATE(x USING LATIN_TO_UNICODE WITH ERROR)")
def test_update(self):
self.validate_all(

10
tests/dialects/test_tsql.py
View file

@ -1,7 +1,7 @@
from sqlglot import exp, parse, parse_one
from tests.dialects.test_dialect import Validator
from sqlglot.errors import ParseError, UnsupportedError
from sqlglot.optimizer.annotate_types import annotate_types
from tests.dialects.test_dialect import Validator
class TestTSQL(Validator):
@ -962,7 +962,8 @@ FROM (
ProductID
) AS src(ProductID, OrderQty)
ON pi.ProductID = src.ProductID
WHEN MATCHED AND pi.Quantity - src.OrderQty >= 0 THEN UPDATE SET pi.Quantity = pi.Quantity - src.OrderQty
WHEN MATCHED AND pi.Quantity - src.OrderQty >= 0 THEN UPDATE SET
pi.Quantity = pi.Quantity - src.OrderQty
WHEN MATCHED AND pi.Quantity - src.OrderQty <= 0 THEN DELETE
OUTPUT $action, Inserted.ProductID, Inserted.LocationID, Inserted.Quantity AS NewQty, Deleted.Quantity AS PreviousQty
) AS Changes(Action, ProductID, LocationID, NewQty, PreviousQty)
@ -2242,3 +2243,8 @@ FROM OPENJSON(@json) WITH (
"tsql": "SELECT DATETRUNC(YEAR, CAST('foo1' AS DATE))",
},
)
def test_collation_parse(self):
self.validate_identity("ALTER TABLE a ALTER COLUMN b CHAR(10) COLLATE abc").assert_is(
exp.Alter
).args.get("actions")[0].args.get("collate").this.assert_is(exp.Var)

119
tests/fixtures/optimizer/annotate_functions.sql vendored
View file

@ -100,6 +100,117 @@ STRING;
RPAD(tbl.str_col, 1, tbl.str_col);
STRING;
# dialect: hive, spark2, spark, databricks
IF(cond, tbl.double_col, tbl.bigint_col);
DOUBLE;
# dialect: hive, spark2, spark, databricks
IF(cond, tbl.bigint_col, tbl.double_col);
DOUBLE;
# dialect: hive, spark2, spark
IF(cond, tbl.double_col, tbl.str_col);
STRING;
# dialect: hive, spark2, spark
IF(cond, tbl.str_col, tbl.double_col);
STRING;
# dialect: databricks
IF(cond, tbl.str_col, tbl.double_col);
DOUBLE;
# dialect: databricks
IF(cond, tbl.double_col, tbl.str_col);
DOUBLE;
# dialect: hive, spark2, spark
IF(cond, tbl.date_col, tbl.str_col);
STRING;
# dialect: hive, spark2, spark
IF(cond, tbl.str_col, tbl.date_col);
STRING;
# dialect: databricks
IF(cond, tbl.date_col, tbl.str_col);
DATE;
# dialect: databricks
IF(cond, tbl.str_col, tbl.date_col);
DATE;
# dialect: hive, spark2, spark, databricks
IF(cond, tbl.date_col, tbl.timestamp_col);
TIMESTAMP;
# dialect: hive, spark2, spark, databricks
IF(cond, tbl.timestamp_col, tbl.date_col);
TIMESTAMP;
# dialect: hive, spark2, spark, databricks
IF(cond, NULL, tbl.str_col);
STRING;
# dialect: hive, spark2, spark, databricks
IF(cond, tbl.str_col, NULL);
STRING;
# dialect: hive, spark2, spark
COALESCE(tbl.str_col, tbl.date_col, tbl.bigint_col);
STRING;
# dialect: hive, spark2, spark
COALESCE(tbl.date_col, tbl.str_col, tbl.bigint_col);
STRING;
# dialect: hive, spark2, spark
COALESCE(tbl.date_col, tbl.bigint_col, tbl.str_col);
STRING;
# dialect: hive, spark2, spark
COALESCE(tbl.str_col, tbl.date_col, tbl.bigint_col);
STRING;
# dialect: hive, spark2, spark
COALESCE(tbl.date_col, tbl.str_col, tbl.bigint_col);
STRING;
# dialect: hive, spark2, spark
COALESCE(tbl.date_col, NULL, tbl.bigint_col, tbl.str_col);
STRING;
# dialect: databricks
COALESCE(tbl.str_col, tbl.bigint_col);
BIGINT;
# dialect: databricks
COALESCE(tbl.bigint_col, tbl.str_col);
BIGINT;
# dialect: databricks
COALESCE(tbl.str_col, NULL, tbl.bigint_col);
BIGINT;
# dialect: databricks
COALESCE(tbl.bigint_col, NULL, tbl.str_col);
BIGINT;
# dialect: databricks
COALESCE(tbl.bool_col, tbl.str_col);
BOOLEAN;
# dialect: hive, spark2, spark
COALESCE(tbl.interval_col, tbl.str_col);
STRING;
# dialect: databricks
COALESCE(tbl.interval_col, tbl.str_col);
INTERVAL;
# dialect: databricks
COALESCE(tbl.bin_col, tbl.str_col);
BINARY;
--------------------------------------
-- BigQuery
@ -205,6 +316,14 @@ STRING;
CONCAT(tbl.bin_col, tbl.bin_col);
BINARY;
# dialect: bigquery
CONCAT(0, tbl.str_col);
STRING;
# dialect: bigquery
CONCAT(tbl.str_col, 0);
STRING;
# dialect: bigquery
LEFT(tbl.str_col, 1);
STRING;

5
tests/fixtures/optimizer/merge_subqueries.sql vendored
View file

@ -329,6 +329,11 @@ FROM
t1;
SELECT x.a AS a, x.b AS b, ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) AS row_num FROM x AS x ORDER BY x.a, x.b, row_num;
# title: Keep ORDER BY
# execute: false
WITH t AS (SELECT t1.x AS x, t1.y AS y, t2.a AS a, t2.b AS b FROM t1 AS t1(x, y) CROSS JOIN t2 AS t2(a, b) ORDER BY t2.a) SELECT t.x AS x, t.y AS y, t.a AS a, t.b AS b FROM t AS t;
SELECT t1.x AS x, t1.y AS y, t2.a AS a, t2.b AS b FROM t1 AS t1(x, y) CROSS JOIN t2 AS t2(a, b) ORDER BY t2.a;
# title: Don't merge window functions, inner table is aliased in outer query
with t1 as (
SELECT

8
tests/fixtures/pretty.sql vendored
View file

@ -449,3 +449,11 @@ SELECT
FROM foo
WHERE
1 = 1 AND /* first comment */ foo.a /* second comment */ = 1;
MERGE INTO t USING s ON t.id = s.id WHEN MATCHED THEN UPDATE SET status = s.status, amount = s.amount;
MERGE INTO t
USING s
ON t.id = s.id
WHEN MATCHED THEN UPDATE SET
status = s.status,
amount = s.amount;

10
tests/test_executor.py
View file

@ -885,3 +885,13 @@ class TestExecutor(unittest.TestCase):
"avg_bill_length",
"avg_bill_depth",
]
def test_table_to_pylist(self):
columns = ["id", "product", "price"]
rows = [[1, "Shirt", 20.0], [2, "Shoes", 60.0]]
table = Table(columns=columns, rows=rows)
expected = [
{"id": 1, "product": "Shirt", "price": 20.0},
{"id": 2, "product": "Shoes", "price": 60.0},
]
self.assertEqual(table.to_pylist(), expected)

28
tests/test_optimizer.py
View file

@ -891,7 +891,17 @@ FROM READ_CSV('tests/fixtures/optimizer/tpc-h/nation.csv.gz', 'delimiter', '|')
def test_annotate_funcs(self):
test_schema = {
"tbl": {"bin_col": "BINARY", "str_col": "STRING", "bignum_col": "BIGNUMERIC"}
"tbl": {
"bin_col": "BINARY",
"str_col": "STRING",
"bignum_col": "BIGNUMERIC",
"date_col": "DATE",
"timestamp_col": "TIMESTAMP",
"double_col": "DOUBLE",
"bigint_col": "BIGINT",
"bool_col": "BOOLEAN",
"interval_col": "INTERVAL",
}
}
for i, (meta, sql, expected) in enumerate(
@ -1552,3 +1562,19 @@ FROM READ_CSV('tests/fixtures/optimizer/tpc-h/nation.csv.gz', 'delimiter', '|')
self.assertEqual(4, normalization_distance(gen_expr(2), max_=100))
self.assertEqual(18, normalization_distance(gen_expr(3), max_=100))
self.assertEqual(110, normalization_distance(gen_expr(10), max_=100))
def test_manually_annotate_snowflake(self):
dialect = "snowflake"
schema = {
"SCHEMA": {
"TBL": {"COL": "INT", "col2": "VARCHAR"},
}
}
example_query = 'SELECT * FROM "SCHEMA"."TBL"'
expression = parse_one(example_query, dialect=dialect)
qual = optimizer.qualify.qualify(expression, schema=schema, dialect=dialect)
annotated = optimizer.annotate_types.annotate_types(qual, schema=schema, dialect=dialect)
self.assertTrue(annotated.selects[0].is_type("INT"))
self.assertTrue(annotated.selects[1].is_type("VARCHAR"))

10
tests/test_tokens.py
View file

@ -67,9 +67,19 @@ x"""
tokens = Tokenizer().tokenize("SELECT\r\n 1,\r\n 2")
self.assertEqual(tokens[0].line, 1)
self.assertEqual(tokens[0].col, 6)
self.assertEqual(tokens[1].line, 2)
self.assertEqual(tokens[1].col, 3)
self.assertEqual(tokens[2].line, 2)
self.assertEqual(tokens[2].col, 4)
self.assertEqual(tokens[3].line, 3)
self.assertEqual(tokens[3].col, 3)
tokens = Tokenizer().tokenize(" SELECT\n 100")
self.assertEqual(tokens[0].line, 1)
self.assertEqual(tokens[0].col, 8)
self.assertEqual(tokens[1].line, 2)
self.assertEqual(tokens[1].col, 7)
def test_crlf(self):
tokens = Tokenizer().tokenize("SELECT a\r\nFROM b")

13
tests/test_transforms.py
View file

@ -5,6 +5,7 @@ from sqlglot.transforms import (
eliminate_distinct_on,
eliminate_join_marks,
eliminate_qualify,
eliminate_window_clause,
remove_precision_parameterized_types,
unalias_group,
)
@ -272,3 +273,15 @@ class TestTransforms(unittest.TestCase):
tree.sql(dialect=dialect)
== "SELECT a.id FROM a LEFT JOIN b ON a.id = b.id AND b.d = const"
)
def test_eliminate_window_clause(self):
self.validate(
eliminate_window_clause,
"SELECT purchases, LAST_VALUE(item) OVER (d) AS most_popular FROM Produce WINDOW a AS (PARTITION BY purchases), b AS (a ORDER BY purchases), c AS (b ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING), d AS (c)",
"SELECT purchases, LAST_VALUE(item) OVER (PARTITION BY purchases ORDER BY purchases ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING) AS most_popular FROM Produce",
)
self.validate(
eliminate_window_clause,
"SELECT LAST_VALUE(c) OVER (a) AS c2 FROM (SELECT LAST_VALUE(i) OVER (a) AS c FROM p WINDOW a AS (PARTITION BY x)) AS q(c) WINDOW a AS (PARTITION BY y)",
"SELECT LAST_VALUE(c) OVER (PARTITION BY y) AS c2 FROM (SELECT LAST_VALUE(i) OVER (PARTITION BY x) AS c FROM p) AS q(c)",
)