dbcli/sqlglot
1
0
Fork 0
Code Issues Activity

Adding upstream version 19.0.1.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-02-13 21:15:38 +01:00
parent 03001ce1e6
commit 6a89523da4
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
91 changed files with 45416 additions and 43096 deletions
Download patch file Download diff file Expand all files Collapse all files

66
CHANGELOG.md
View file

@ -1,6 +1,70 @@
Changelog Changelog
========= =========
## [v19.0.0] - 2023-10-30
### :boom: BREAKING CHANGES
- due to [`7a6da28`](https://github.com/tobymao/sqlglot/commit/7a6da28907ef519b954bc813fe180c306fe5006b) - generator now always copies, making transforms much simpler *(PR [#2477](https://github.com/tobymao/sqlglot/pull/2477) by [@tobymao](https://github.com/tobymao))*:
generator now always copies, making transforms much simpler (#2477)
### :sparkles: New Features
- [`542ea6c`](https://github.com/tobymao/sqlglot/commit/542ea6c196679ecb28f16f28c8fe1d62bf9f82e1) - **mysql**: convert full outer join to union with left/right outer joins *(PR [#2461](https://github.com/tobymao/sqlglot/pull/2461) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- :arrow_lower_right: *addresses issue [#2458](undefined) opened by [@treysp](https://github.com/treysp)*
- [`b3990bc`](https://github.com/tobymao/sqlglot/commit/b3990bc74131ff537303eb24608d40f7456a9ee9) - **postgres**: support WITH ORDINALITY in table functions *(PR [#2465](https://github.com/tobymao/sqlglot/pull/2465) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- :arrow_lower_right: *addresses issue [#1619](undefined) opened by [@deschman](https://github.com/deschman)*
- [`c3852db`](https://github.com/tobymao/sqlglot/commit/c3852db1f873b2ed4096d582aa334723512a545d) - **postgres**: add support for the PARTITION OF property in CREATE *(PR [#2476](https://github.com/tobymao/sqlglot/pull/2476) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- :arrow_lower_right: *addresses issue [#2469](undefined) opened by [@judahrand](https://github.com/judahrand)*
### :bug: Bug Fixes
- [`4f9cb22`](https://github.com/tobymao/sqlglot/commit/4f9cb2297119fd61dc4ab88b76c4db5c46e4ece6) - facilitate transpilation of Redshift's LISTAGG *(PR [#2460](https://github.com/tobymao/sqlglot/pull/2460) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- [`81ab80a`](https://github.com/tobymao/sqlglot/commit/81ab80a745531fa345e7ea45a356334c68a908a0) - **mysql**: str_to_date for datetime *(PR [#2473](https://github.com/tobymao/sqlglot/pull/2473) by [@barakalon](https://github.com/barakalon))*
- [`a1252d8`](https://github.com/tobymao/sqlglot/commit/a1252d8ba7d2394bbb14ccd42d835da8cd4eb740) - **teradata**: add eq, minus abbreviations fixes [#2474](https://github.com/tobymao/sqlglot/pull/2474) *(commit by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- [`e6f31d6`](https://github.com/tobymao/sqlglot/commit/e6f31d65825d6e3324b4564d202e610d3068e263) - **snowflake**: don't add time format in TO_TIMESTAMP if not supplied *(PR [#2475](https://github.com/tobymao/sqlglot/pull/2475) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- [`2307910`](https://github.com/tobymao/sqlglot/commit/23079105af1bdcbd849d813b402ee1a3b55fdacd) - **optimizer**: make normalize_identifiers identifier conversion more lenient *(PR [#2478](https://github.com/tobymao/sqlglot/pull/2478) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- [`ed5c559`](https://github.com/tobymao/sqlglot/commit/ed5c5593a6316da81695b1bfaad90b465acb99ba) - **snowflake**: avoid crash on OBJECT_CONSTRUCT without arguments *(PR [#2482](https://github.com/tobymao/sqlglot/pull/2482) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- :arrow_lower_right: *fixes issue [#2479](undefined) opened by [@wedotech-ashley](https://github.com/wedotech-ashley)*
- [`df0a698`](https://github.com/tobymao/sqlglot/commit/df0a69842daf3ec957effba206b79e27a97c91b4) - **tsql**: add 'dddd' to '%A' time mapping *(PR [#2483](https://github.com/tobymao/sqlglot/pull/2483) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- :arrow_lower_right: *fixes issue [#2481](undefined) opened by [@SudarshanVS](https://github.com/SudarshanVS)*
### :recycle: Refactors
- [`7a6da28`](https://github.com/tobymao/sqlglot/commit/7a6da28907ef519b954bc813fe180c306fe5006b) - generator now always copies, making transforms much simpler *(PR [#2477](https://github.com/tobymao/sqlglot/pull/2477) by [@tobymao](https://github.com/tobymao))*
## [v18.17.0] - 2023-10-25
### :boom: BREAKING CHANGES
- due to [`c8e87b6`](https://github.com/tobymao/sqlglot/commit/c8e87b6cab18924b6fa307252f19e83eef4f2f03) - unnabreviate units, e.g. ms to millisecond *(PR [#2451](https://github.com/tobymao/sqlglot/pull/2451) by [@GeorgeSittas](https://github.com/GeorgeSittas))*:
unnabreviate units, e.g. ms to millisecond (#2451)
### :sparkles: New Features
- [`7ded253`](https://github.com/tobymao/sqlglot/commit/7ded2536f348a0470af044b56fa0fd6cfb09bb0d) - add support for {fn ...} function syntax *(PR [#2447](https://github.com/tobymao/sqlglot/pull/2447) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- [`ee18756`](https://github.com/tobymao/sqlglot/commit/ee18756e8779f0267025e63789bddf99cdf631e0) - **snowflake**: register APPROX_TOP_K as AggFunc *(PR [#2450](https://github.com/tobymao/sqlglot/pull/2450) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- :arrow_lower_right: *addresses issue [#2449](undefined) opened by [@yogevyuval](https://github.com/yogevyuval)*
- [`57b744a`](https://github.com/tobymao/sqlglot/commit/57b744a431614818cd84f65a8d6105655cd462a4) - **optimizer**: infer bracket type *(PR [#2441](https://github.com/tobymao/sqlglot/pull/2441) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- [`64b18e6`](https://github.com/tobymao/sqlglot/commit/64b18e6555865e447723e8e4619e9eba399e7cee) - add ArgMax, ArgMin expressions, fix their transpilation *(PR [#2454](https://github.com/tobymao/sqlglot/pull/2454) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- :arrow_lower_right: *addresses issue [#2453](undefined) opened by [@erikcw](https://github.com/erikcw)*
- [`959fa92`](https://github.com/tobymao/sqlglot/commit/959fa9212ee698029eb6e1957d4f2d22590fccb0) - **lineage**: terminal table node for select star *(PR [#2456](https://github.com/tobymao/sqlglot/pull/2456) by [@edpaget](https://github.com/edpaget))*
### :bug: Bug Fixes
- [`e11ecaf`](https://github.com/tobymao/sqlglot/commit/e11ecaf2e0c5225df7ea5ca6e5622be1e94b2625) - **clickhouse**: treat CURRENT_DATE as a function *(PR [#2439](https://github.com/tobymao/sqlglot/pull/2439) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- :arrow_lower_right: *fixes issue [#2438](undefined) opened by [@samuelcolvin](https://github.com/samuelcolvin)*
- [`fdb1668`](https://github.com/tobymao/sqlglot/commit/fdb166801144b721677d23c195e5bd3d35ee8841) - improve bracket parsing error, set Slice type to Unknown *(PR [#2440](https://github.com/tobymao/sqlglot/pull/2440) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- :arrow_lower_right: *fixes issue [#2437](undefined) opened by [@samuelcolvin](https://github.com/samuelcolvin)*
- [`546eb54`](https://github.com/tobymao/sqlglot/commit/546eb545f2f6a37408c41d013089f2f29e922eb3) - **redshift**: don't rewrite JSON_PARSE to PARSE_JSON *(commit by [@purcell](https://github.com/purcell))*
- [`00e3515`](https://github.com/tobymao/sqlglot/commit/00e3515ff3157105d84f21f56cfb64bae3267640) - **parser**: treat 'use' as a valid identifier token *(PR [#2446](https://github.com/tobymao/sqlglot/pull/2446) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- :arrow_lower_right: *fixes issue [#2444](undefined) opened by [@CBQu](https://github.com/CBQu)*
- [`c8e87b6`](https://github.com/tobymao/sqlglot/commit/c8e87b6cab18924b6fa307252f19e83eef4f2f03) - unnabreviate units, e.g. ms to millisecond *(PR [#2451](https://github.com/tobymao/sqlglot/pull/2451) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- :arrow_lower_right: *fixes issue [#2448](undefined) opened by [@samuelcolvin](https://github.com/samuelcolvin)*
- [`11bfc75`](https://github.com/tobymao/sqlglot/commit/11bfc7518bb39938661c55d2de54ca0cbd042c6a) - **lineage**: handle unions with SELECT STAR *(PR [#2452](https://github.com/tobymao/sqlglot/pull/2452) by [@edpaget](https://github.com/edpaget))*
- [`b856477`](https://github.com/tobymao/sqlglot/commit/b856477200faf552441a8d27adb5cff4b83dda65) - add table alias in unnest with columns only *(PR [#2457](https://github.com/tobymao/sqlglot/pull/2457) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
- :arrow_lower_right: *fixes issue [#2455](undefined) opened by [@bgedik](https://github.com/bgedik)*
### :wrench: Chores
- [`927f5aa`](https://github.com/tobymao/sqlglot/commit/927f5aaf20155fe5c7e0a1c466eae19c59913045) - add docstrings in rest of transforms *(commit by [@GeorgeSittas](https://github.com/GeorgeSittas))*
## [v18.16.1] - 2023-10-21 ## [v18.16.1] - 2023-10-21
### :sparkles: New Features ### :sparkles: New Features
- [`16fb83f`](https://github.com/tobymao/sqlglot/commit/16fb83ff3f9a2bd77ce3edfa6f4932916a033d4c) - **bigquery**: default collate closes [#2434](https://github.com/tobymao/sqlglot/pull/2434) *(commit by [@tobymao](https://github.com/tobymao))* - [`16fb83f`](https://github.com/tobymao/sqlglot/commit/16fb83ff3f9a2bd77ce3edfa6f4932916a033d4c) - **bigquery**: default collate closes [#2434](https://github.com/tobymao/sqlglot/pull/2434) *(commit by [@tobymao](https://github.com/tobymao))*
@ -1778,3 +1842,5 @@ Changelog
[v18.15.1]: https://github.com/tobymao/sqlglot/compare/v18.15.0...v18.15.1 [v18.15.1]: https://github.com/tobymao/sqlglot/compare/v18.15.0...v18.15.1
[v18.16.0]: https://github.com/tobymao/sqlglot/compare/v18.15.1...v18.16.0 [v18.16.0]: https://github.com/tobymao/sqlglot/compare/v18.15.1...v18.16.0
[v18.16.1]: https://github.com/tobymao/sqlglot/compare/v18.16.0...v18.16.1 [v18.16.1]: https://github.com/tobymao/sqlglot/compare/v18.16.0...v18.16.1
[v18.17.0]: https://github.com/tobymao/sqlglot/compare/v18.16.1...v18.17.0
[v19.0.0]: https://github.com/tobymao/sqlglot/compare/v18.17.0...v19.0.0

2
README.md
View file

@ -1,6 +1,6 @@
![SQLGlot logo](sqlglot.svg) ![SQLGlot logo](sqlglot.svg)
SQLGlot is a no-dependency SQL parser, transpiler, optimizer, and engine. It can be used to format SQL or translate between [20 different dialects](https://github.com/tobymao/sqlglot/blob/main/sqlglot/dialects/__init__.py) like [DuckDB](https://duckdb.org/), [Presto](https://prestodb.io/), [Spark](https://spark.apache.org/), [Snowflake](https://www.snowflake.com/en/), and [BigQuery](https://cloud.google.com/bigquery/). It aims to read a wide variety of SQL inputs and output syntactically and semantically correct SQL in the targeted dialects. SQLGlot is a no-dependency SQL parser, transpiler, optimizer, and engine. It can be used to format SQL or translate between [20 different dialects](https://github.com/tobymao/sqlglot/blob/main/sqlglot/dialects/__init__.py) like [DuckDB](https://duckdb.org/), [Presto](https://prestodb.io/) / [Trino](https://trino.io/), [Spark](https://spark.apache.org/) / [Databricks](https://www.databricks.com/), [Snowflake](https://www.snowflake.com/en/), and [BigQuery](https://cloud.google.com/bigquery/). It aims to read a wide variety of SQL inputs and output syntactically and semantically correct SQL in the targeted dialects.
It is a very comprehensive generic SQL parser with a robust [test suite](https://github.com/tobymao/sqlglot/blob/main/tests/). It is also quite [performant](#benchmarks), while being written purely in Python. It is a very comprehensive generic SQL parser with a robust [test suite](https://github.com/tobymao/sqlglot/blob/main/tests/). It is also quite [performant](#benchmarks), while being written purely in Python.

2
docs/search.js
View file

File diff suppressed because one or more lines are too long

6
docs/sqlglot.html
View file

@ -96,7 +96,7 @@
<div class="docstring"><p><img src="sqlglot.svg" alt="SQLGlot logo" /></p> <div class="docstring"><p><img src="sqlglot.svg" alt="SQLGlot logo" /></p>
<p>SQLGlot is a no-dependency SQL parser, transpiler, optimizer, and engine. It can be used to format SQL or translate between <a href="https://github.com/tobymao/sqlglot/blob/main/sqlglot/dialects/__init__.py">20 different dialects</a> like <a href="https://duckdb.org/">DuckDB</a>, <a href="https://prestodb.io/">Presto</a>, <a href="https://spark.apache.org/">Spark</a>, <a href="https://www.snowflake.com/en/">Snowflake</a>, and <a href="https://cloud.google.com/bigquery/">BigQuery</a>. It aims to read a wide variety of SQL inputs and output syntactically and semantically correct SQL in the targeted dialects.</p> <p>SQLGlot is a no-dependency SQL parser, transpiler, optimizer, and engine. It can be used to format SQL or translate between <a href="https://github.com/tobymao/sqlglot/blob/main/sqlglot/dialects/__init__.py">20 different dialects</a> like <a href="https://duckdb.org/">DuckDB</a>, <a href="https://prestodb.io/">Presto</a> / <a href="https://trino.io/">Trino</a>, <a href="https://spark.apache.org/">Spark</a> / <a href="https://www.databricks.com/">Databricks</a>, <a href="https://www.snowflake.com/en/">Snowflake</a>, and <a href="https://cloud.google.com/bigquery/">BigQuery</a>. It aims to read a wide variety of SQL inputs and output syntactically and semantically correct SQL in the targeted dialects.</p>
<p>It is a very comprehensive generic SQL parser with a robust <a href="https://github.com/tobymao/sqlglot/blob/main/tests/">test suite</a>. It is also quite <a href="#benchmarks">performant</a>, while being written purely in Python.</p> <p>It is a very comprehensive generic SQL parser with a robust <a href="https://github.com/tobymao/sqlglot/blob/main/tests/">test suite</a>. It is also quite <a href="#benchmarks">performant</a>, while being written purely in Python.</p>
@ -833,7 +833,7 @@ make check # Full test suite &amp; linter checks
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">write</span> <span class="o">=</span> <span class="p">(</span><span class="n">read</span> <span class="k">if</span> <span class="n">write</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">write</span><span class="p">)</span> <span class="k">if</span> <span class="n">identity</span> <span class="k">else</span> <span class="n">write</span> </span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">write</span> <span class="o">=</span> <span class="p">(</span><span class="n">read</span> <span class="k">if</span> <span class="n">write</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">write</span><span class="p">)</span> <span class="k">if</span> <span class="n">identity</span> <span class="k">else</span> <span class="n">write</span>
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="k">return</span> <span class="p">[</span> </span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="k">return</span> <span class="p">[</span>
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">write</span><span class="p">)()</span><span class="o">.</span><span class="n">generate</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span> </span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">write</span><span class="p">)()</span><span class="o">.</span><span class="n">generate</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span> <span class="k">if</span> <span class="n">expression</span> <span class="k">else</span> <span class="s2">&quot;&quot;</span>
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">parse</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">read</span><span class="p">,</span> <span class="n">error_level</span><span class="o">=</span><span class="n">error_level</span><span class="p">)</span> </span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">parse</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">read</span><span class="p">,</span> <span class="n">error_level</span><span class="o">=</span><span class="n">error_level</span><span class="p">)</span>
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="p">]</span> </span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="p">]</span>
</span></pre></div> </span></pre></div>
@ -1037,7 +1037,7 @@ make check # Full test suite &amp; linter checks
</span><span id="transpile-159"><a href="#transpile-159"><span class="linenos">159</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="transpile-159"><a href="#transpile-159"><span class="linenos">159</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="transpile-160"><a href="#transpile-160"><span class="linenos">160</span></a> <span class="n">write</span> <span class="o">=</span> <span class="p">(</span><span class="n">read</span> <span class="k">if</span> <span class="n">write</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">write</span><span class="p">)</span> <span class="k">if</span> <span class="n">identity</span> <span class="k">else</span> <span class="n">write</span> </span><span id="transpile-160"><a href="#transpile-160"><span class="linenos">160</span></a> <span class="n">write</span> <span class="o">=</span> <span class="p">(</span><span class="n">read</span> <span class="k">if</span> <span class="n">write</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">write</span><span class="p">)</span> <span class="k">if</span> <span class="n">identity</span> <span class="k">else</span> <span class="n">write</span>
</span><span id="transpile-161"><a href="#transpile-161"><span class="linenos">161</span></a> <span class="k">return</span> <span class="p">[</span> </span><span id="transpile-161"><a href="#transpile-161"><span class="linenos">161</span></a> <span class="k">return</span> <span class="p">[</span>
</span><span id="transpile-162"><a href="#transpile-162"><span class="linenos">162</span></a> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">write</span><span class="p">)()</span><span class="o">.</span><span class="n">generate</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span> </span><span id="transpile-162"><a href="#transpile-162"><span class="linenos">162</span></a> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">write</span><span class="p">)()</span><span class="o">.</span><span class="n">generate</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="o">**</span><span class="n">opts</span><span class="p">)</span> <span class="k">if</span> <span class="n">expression</span> <span class="k">else</span> <span class="s2">&quot;&quot;</span>
</span><span id="transpile-163"><a href="#transpile-163"><span class="linenos">163</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">parse</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">read</span><span class="p">,</span> <span class="n">error_level</span><span class="o">=</span><span class="n">error_level</span><span class="p">)</span> </span><span id="transpile-163"><a href="#transpile-163"><span class="linenos">163</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">parse</span><span class="p">(</span><span class="n">sql</span><span class="p">,</span> <span class="n">read</span><span class="p">,</span> <span class="n">error_level</span><span class="o">=</span><span class="n">error_level</span><span class="p">)</span>
</span><span id="transpile-164"><a href="#transpile-164"><span class="linenos">164</span></a> <span class="p">]</span> </span><span id="transpile-164"><a href="#transpile-164"><span class="linenos">164</span></a> <span class="p">]</span>
</span></pre></div> </span></pre></div>

8
docs/sqlglot/_version.html
View file

@ -76,8 +76,8 @@
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span> </span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="n">__version_tuple__</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span> </span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="n">version_tuple</span><span class="p">:</span> <span class="n">VERSION_TUPLE</span>
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a> </span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a>
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;18.16.1&#39;</span> </span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">&#39;19.0.0&#39;</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">18</span><span class="p">,</span> <span class="mi">16</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span> </span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">19</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
</span></pre></div> </span></pre></div>
@ -97,7 +97,7 @@
<section id="version"> <section id="version">
<div class="attr variable"> <div class="attr variable">
<span class="name">version</span><span class="annotation">: str</span> = <span class="name">version</span><span class="annotation">: str</span> =
<span class="default_value">&#39;18.16.1&#39;</span> <span class="default_value">&#39;19.0.0&#39;</span>
</div> </div>
@ -109,7 +109,7 @@
<section id="version_tuple"> <section id="version_tuple">
<div class="attr variable"> <div class="attr variable">
<span class="name">version_tuple</span><span class="annotation">: object</span> = <span class="name">version_tuple</span><span class="annotation">: object</span> =
<span class="default_value">(18, 16, 1)</span> <span class="default_value">(19, 0, 0)</span>
</div> </div>

40
docs/sqlglot/dataframe/sql.html
View file

@ -777,7 +777,7 @@
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">createDataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">data</span><span class="p">:</span> <span class="n">Sequence</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452315022240&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452315022240&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="n">Tuple</span><span class="p">]]</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452314825728&#39;</span><span class="o">&gt;</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">samplingRatio</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">verifySchema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span> <span class="name">createDataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">data</span><span class="p">:</span> <span class="n">Sequence</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452569658896&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452569658896&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="n">Tuple</span><span class="p">]]</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452569891840&#39;</span><span class="o">&gt;</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">samplingRatio</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">verifySchema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<label class="view-source-button" for="SparkSession.createDataFrame-view-source"><span>View Source</span></label> <label class="view-source-button" for="SparkSession.createDataFrame-view-source"><span>View Source</span></label>
@ -1829,7 +1829,7 @@
<input id="DataFrame.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="DataFrame.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<div class="attr function"> <div class="attr function">
<span class="name">DataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">spark</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452319856048&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Select">sqlglot.expressions.Select</a></span>,</span><span class="param"> <span class="n">branch_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">sequence_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">last_op</span><span class="p">:</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">dataframe</span><span class="o">.</span><span class="n">sql</span><span class="o">.</span><span class="n">operations</span><span class="o">.</span><span class="n">Operation</span> <span class="o">=</span> <span class="o">&lt;</span><span class="n">Operation</span><span class="o">.</span><span class="n">INIT</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">pending_hints</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">output_expression_container</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452320025888&#39;</span><span class="o">&gt;</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span>)</span> <span class="name">DataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">spark</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452574562864&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Select">sqlglot.expressions.Select</a></span>,</span><span class="param"> <span class="n">branch_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">sequence_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">last_op</span><span class="p">:</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">dataframe</span><span class="o">.</span><span class="n">sql</span><span class="o">.</span><span class="n">operations</span><span class="o">.</span><span class="n">Operation</span> <span class="o">=</span> <span class="o">&lt;</span><span class="n">Operation</span><span class="o">.</span><span class="n">INIT</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">pending_hints</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">output_expression_container</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452574768448&#39;</span><span class="o">&gt;</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span>)</span>
<label class="view-source-button" for="DataFrame.__init__-view-source"><span>View Source</span></label> <label class="view-source-button" for="DataFrame.__init__-view-source"><span>View Source</span></label>
@ -2018,7 +2018,7 @@
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">sql</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452315728432&#39;</span><span class="o">&gt;</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">optimize</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span>:</span></span> <span class="name">sql</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">dialect</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452570319408&#39;</span><span class="o">&gt;</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">optimize</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span>:</span></span>
<label class="view-source-button" for="DataFrame.sql-view-source"><span>View Source</span></label> <label class="view-source-button" for="DataFrame.sql-view-source"><span>View Source</span></label>
@ -2773,7 +2773,7 @@ is unlikely to come up.</p>
<div class="decorator">@operation(Operation.FROM)</div> <div class="decorator">@operation(Operation.FROM)</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">fillna</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452315832640&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span> <span class="name">fillna</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452568654144&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<label class="view-source-button" for="DataFrame.fillna-view-source"><span>View Source</span></label> <label class="view-source-button" for="DataFrame.fillna-view-source"><span>View Source</span></label>
@ -2842,7 +2842,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@operation(Operation.FROM)</div> <div class="decorator">@operation(Operation.FROM)</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">replace</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">to_replace</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">Dict</span><span class="p">]</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n">Collection</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452313861280&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452313861280&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span> <span class="name">replace</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">to_replace</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">Dict</span><span class="p">]</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n">Collection</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452569009312&#39;</span><span class="o">&gt;</span><span class="p">],</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452569009312&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<label class="view-source-button" for="DataFrame.replace-view-source"><span>View Source</span></label> <label class="view-source-button" for="DataFrame.replace-view-source"><span>View Source</span></label>
@ -3047,7 +3047,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@operation(Operation.NO_OP)</div> <div class="decorator">@operation(Operation.NO_OP)</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">repartition</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">numPartitions</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452313784256&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452314075040&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span> <span class="name">repartition</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">numPartitions</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452569096128&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452569223072&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">DataFrame</a></span>:</span></span>
<label class="view-source-button" for="DataFrame.repartition-view-source"><span>View Source</span></label> <label class="view-source-button" for="DataFrame.repartition-view-source"><span>View Source</span></label>
@ -3765,7 +3765,7 @@ and check if it matches the type of the value provided. If not then make it null
<input id="Column.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="Column.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<div class="attr function"> <div class="attr function">
<span class="name">Column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452317817648&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span>)</span> <span class="name">Column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452570822352&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span>)</span>
<label class="view-source-button" for="Column.__init__-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.__init__-view-source"><span>View Source</span></label>
@ -3809,7 +3809,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@classmethod</div> <div class="decorator">@classmethod</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">ensure_col</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452314678176&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span> <span class="name">ensure_col</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452569549264&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.ensure_col-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.ensure_col-view-source"><span>View Source</span></label>
@ -3830,7 +3830,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@classmethod</div> <div class="decorator">@classmethod</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">ensure_cols</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">args</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452314457488&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="n"><a href="#Column">Column</a></span><span class="p">]</span>:</span></span> <span class="name">ensure_cols</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">args</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452569523600&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="n"><a href="#Column">Column</a></span><span class="p">]</span>:</span></span>
<label class="view-source-button" for="Column.ensure_cols-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.ensure_cols-view-source"><span>View Source</span></label>
@ -3851,7 +3851,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@classmethod</div> <div class="decorator">@classmethod</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">invoke_anonymous_function</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452314721072&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="n">func_name</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452314417088&#39;</span><span class="o">&gt;</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span> <span class="name">invoke_anonymous_function</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452569438608&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="n">func_name</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452569350928&#39;</span><span class="o">&gt;</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.invoke_anonymous_function-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.invoke_anonymous_function-view-source"><span>View Source</span></label>
@ -3878,7 +3878,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="decorator">@classmethod</div> <div class="decorator">@classmethod</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">invoke_expression_over_column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452314715216&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="n">callable_expression</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span> <span class="name">invoke_expression_over_column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452569443312&#39;</span><span class="o">&gt;</span><span class="p">]</span>,</span><span class="param"> <span class="n">callable_expression</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.invoke_expression_over_column-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.invoke_expression_over_column-view-source"><span>View Source</span></label>
@ -3915,7 +3915,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452314588704&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span> <span class="name">binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452567492128&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.binary_op-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.binary_op-view-source"><span>View Source</span></label>
@ -3936,7 +3936,7 @@ and check if it matches the type of the value provided. If not then make it null
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">inverse_binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452314632720&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span> <span class="name">inverse_binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452567568912&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.inverse_binary_op-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.inverse_binary_op-view-source"><span>View Source</span></label>
@ -4502,7 +4502,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">isin</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452314786560&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">Iterable</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452314786560&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">):</span></span> <span class="name">isin</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452567722752&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">Iterable</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452567722752&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">):</span></span>
<label class="view-source-button" for="Column.isin-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.isin-view-source"><span>View Source</span></label>
@ -4523,7 +4523,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">between</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">lowerBound</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452312840752&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">upperBound</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452312878912&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span> <span class="name">between</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">lowerBound</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452567890480&#39;</span><span class="o">&gt;</span>,</span><span class="param"> <span class="n">upperBound</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452567945024&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.between-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.between-view-source"><span>View Source</span></label>
@ -4558,7 +4558,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">over</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">window</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452315623488&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span> <span class="name">over</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">window</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452567986240&#39;</span><span class="o">&gt;</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">Column</a></span>:</span></span>
<label class="view-source-button" for="Column.over-view-source"><span>View Source</span></label> <label class="view-source-button" for="Column.over-view-source"><span>View Source</span></label>
@ -4803,7 +4803,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="decorator">@classmethod</div> <div class="decorator">@classmethod</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452313196160&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452313196160&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span> <span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452568376960&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452568376960&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<label class="view-source-button" for="Window.partitionBy-view-source"><span>View Source</span></label> <label class="view-source-button" for="Window.partitionBy-view-source"><span>View Source</span></label>
@ -4824,7 +4824,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="decorator">@classmethod</div> <div class="decorator">@classmethod</div>
<span class="def">def</span> <span class="def">def</span>
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452313545552&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452313545552&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span> <span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452568432896&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452568432896&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<label class="view-source-button" for="Window.orderBy-view-source"><span>View Source</span></label> <label class="view-source-button" for="Window.orderBy-view-source"><span>View Source</span></label>
@ -5064,7 +5064,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452313094016&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452313094016&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span> <span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452568065808&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452568065808&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<label class="view-source-button" for="WindowSpec.partitionBy-view-source"><span>View Source</span></label> <label class="view-source-button" for="WindowSpec.partitionBy-view-source"><span>View Source</span></label>
@ -5091,7 +5091,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452313178048&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452313178048&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span> <span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452568047552&#39;</span><span class="o">&gt;</span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o">&lt;</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">&#39;140452568047552&#39;</span><span class="o">&gt;</span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">WindowSpec</a></span>:</span></span>
<label class="view-source-button" for="WindowSpec.orderBy-view-source"><span>View Source</span></label> <label class="view-source-button" for="WindowSpec.orderBy-view-source"><span>View Source</span></label>

1814
docs/sqlglot/dialects/bigquery.html
View file

File diff suppressed because one or more lines are too long

2976
docs/sqlglot/dialects/clickhouse.html
View file

File diff suppressed because one or more lines are too long

132
docs/sqlglot/dialects/databricks.html
View file

File diff suppressed because one or more lines are too long

1395
docs/sqlglot/dialects/dialect.html
View file

File diff suppressed because it is too large Load diff

8
docs/sqlglot/dialects/doris.html
View file

File diff suppressed because one or more lines are too long

676
docs/sqlglot/dialects/drill.html
View file

File diff suppressed because one or more lines are too long

1797
docs/sqlglot/dialects/duckdb.html
View file

File diff suppressed because one or more lines are too long

2743
docs/sqlglot/dialects/hive.html
View file

File diff suppressed because one or more lines are too long

4168
docs/sqlglot/dialects/mysql.html
View file

File diff suppressed because one or more lines are too long

12
docs/sqlglot/dialects/oracle.html
View file

File diff suppressed because one or more lines are too long

1745
docs/sqlglot/dialects/postgres.html
View file

File diff suppressed because one or more lines are too long

1102
docs/sqlglot/dialects/presto.html
View file

File diff suppressed because one or more lines are too long

979
docs/sqlglot/dialects/redshift.html
View file

File diff suppressed because one or more lines are too long

1198
docs/sqlglot/dialects/snowflake.html
View file

File diff suppressed because one or more lines are too long

272
docs/sqlglot/dialects/spark.html
View file

File diff suppressed because one or more lines are too long

714
docs/sqlglot/dialects/spark2.html
View file

File diff suppressed because one or more lines are too long

12
docs/sqlglot/dialects/sqlite.html
View file

File diff suppressed because one or more lines are too long

8
docs/sqlglot/dialects/starrocks.html
View file

File diff suppressed because one or more lines are too long

8
docs/sqlglot/dialects/tableau.html
View file

File diff suppressed because one or more lines are too long

1222
docs/sqlglot/dialects/teradata.html
View file

File diff suppressed because one or more lines are too long

6
docs/sqlglot/dialects/trino.html
View file

File diff suppressed because one or more lines are too long

3650
docs/sqlglot/dialects/tsql.html
View file

File diff suppressed because one or more lines are too long

6
docs/sqlglot/executor/python.html
View file

File diff suppressed because one or more lines are too long

17381
docs/sqlglot/expressions.html
View file

File diff suppressed because one or more lines are too long

15114
docs/sqlglot/generator.html
View file

File diff suppressed because one or more lines are too long

1032
docs/sqlglot/helper.html
View file

File diff suppressed because it is too large Load diff

846
docs/sqlglot/lineage.html
View file

@ -224,184 +224,195 @@
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="n">column</span> </span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="n">column</span>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">int</span><span class="p">)</span> </span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">int</span><span class="p">)</span>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="k">else</span> <span class="nb">next</span><span class="p">(</span> </span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="k">else</span> <span class="nb">next</span><span class="p">(</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">i</span> </span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="p">(</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">select</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span> </span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">i</span>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">column</span> </span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">select</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="p">)</span> </span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">column</span> <span class="ow">or</span> <span class="n">select</span><span class="o">.</span><span class="n">is_star</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="p">)</span> </span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="p">),</span>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> </span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="c1"># mypy will not allow a None here, but a negative index should never be returned</span>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">union_scopes</span><span class="p">:</span> </span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="p">)</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="n">to_node</span><span class="p">(</span><span class="n">index</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">s</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">upstream</span><span class="p">)</span> </span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="p">)</span>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> </span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">return</span> <span class="n">upstream</span> </span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">if</span> <span class="n">index</span> <span class="o">==</span> <span class="o">-</span><span class="mi">1</span><span class="p">:</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> </span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;Could not find </span><span class="si">{</span><span class="n">column</span><span class="si">}</span><span class="s2"> in </span><span class="si">{</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">)</span>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="n">subquery</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">unalias</span><span class="p">()</span> </span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> </span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">union_scopes</span><span class="p">:</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">subquery</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span> </span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">to_node</span><span class="p">(</span><span class="n">index</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">s</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">upstream</span><span class="p">)</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">upstream</span> <span class="o">=</span> <span class="n">upstream</span> <span class="ow">or</span> <span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&quot;SUBQUERY&quot;</span><span class="p">,</span> <span class="n">source</span><span class="o">=</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">select</span><span class="p">)</span> </span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="n">scope</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">Scope</span><span class="p">,</span> <span class="n">build_scope</span><span class="p">(</span><span class="n">subquery</span><span class="o">.</span><span class="n">unnest</span><span class="p">()))</span> </span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">return</span> <span class="n">upstream</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> </span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">subquery</span><span class="o">.</span><span class="n">named_selects</span><span class="p">:</span> </span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="n">subquery</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">unalias</span><span class="p">()</span>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="n">to_node</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">scope</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">upstream</span><span class="p">)</span> </span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> </span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">subquery</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="k">return</span> <span class="n">upstream</span> </span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="n">upstream</span> <span class="o">=</span> <span class="n">upstream</span> <span class="ow">or</span> <span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&quot;SUBQUERY&quot;</span><span class="p">,</span> <span class="n">source</span><span class="o">=</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">select</span><span class="p">)</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> </span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">scope</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">Scope</span><span class="p">,</span> <span class="n">build_scope</span><span class="p">(</span><span class="n">subquery</span><span class="o">.</span><span class="n">unnest</span><span class="p">()))</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span> </span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="c1"># For better ergonomics in our node labels, replace the full select with</span> </span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">subquery</span><span class="o">.</span><span class="n">named_selects</span><span class="p">:</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="c1"># a version that has only the column we care about.</span> </span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="n">to_node</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">scope</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">upstream</span><span class="p">)</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="c1"># &quot;x&quot;, SELECT x, y FROM foo</span> </span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="c1"># =&gt; &quot;x&quot;, SELECT x FROM foo</span> </span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="k">return</span> <span class="n">upstream</span>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">append</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span> </span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span> </span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="c1"># For better ergonomics in our node labels, replace the full select with</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> </span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="c1"># a version that has only the column we care about.</span>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="c1"># Create the node for this step in the lineage chain, and attach it to the previous one.</span> </span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="c1"># &quot;x&quot;, SELECT x, y FROM foo</span>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">Node</span><span class="p">(</span> </span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="c1"># =&gt; &quot;x&quot;, SELECT x FROM foo</span>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;</span><span class="si">{</span><span class="n">scope_name</span><span class="si">}</span><span class="s2">.</span><span class="si">{</span><span class="n">column</span><span class="si">}</span><span class="s2">&quot;</span> <span class="k">if</span> <span class="n">scope_name</span> <span class="k">else</span> <span class="nb">str</span><span class="p">(</span><span class="n">column</span><span class="p">),</span> </span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">append</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> </span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">expression</span><span class="o">=</span><span class="n">select</span><span class="p">,</span> </span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span>
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">alias</span> <span class="ow">or</span> <span class="s2">&quot;&quot;</span><span class="p">,</span> </span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a>
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="p">)</span> </span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="c1"># Create the node for this step in the lineage chain, and attach it to the previous one.</span>
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="k">if</span> <span class="n">upstream</span><span class="p">:</span> </span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">Node</span><span class="p">(</span>
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="n">upstream</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">node</span><span class="p">)</span> </span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;</span><span class="si">{</span><span class="n">scope_name</span><span class="si">}</span><span class="s2">.</span><span class="si">{</span><span class="n">column</span><span class="si">}</span><span class="s2">&quot;</span> <span class="k">if</span> <span class="n">scope_name</span> <span class="k">else</span> <span class="nb">str</span><span class="p">(</span><span class="n">column</span><span class="p">),</span>
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> </span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span>
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="c1"># Find all columns that went into creating this one to list their lineage nodes.</span> </span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="n">expression</span><span class="o">=</span><span class="n">select</span><span class="p">,</span>
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="n">source_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">))</span> </span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">alias</span> <span class="ow">or</span> <span class="s2">&quot;&quot;</span><span class="p">,</span>
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> </span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="p">)</span>
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="c1"># If the source is a UDTF find columns used in the UTDF to generate the table</span> </span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="k">if</span> <span class="n">upstream</span><span class="p">:</span>
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">UDTF</span><span class="p">):</span> </span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="n">upstream</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="n">source_columns</span> <span class="o">|=</span> <span class="nb">set</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">))</span> </span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a>
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> </span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="c1"># if the select is a star add all scope sources as downstreams</span>
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">source_columns</span><span class="p">:</span> </span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">is_star</span><span class="p">:</span>
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">c</span><span class="o">.</span><span class="n">table</span> </span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="k">for</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> </span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="n">select</span><span class="o">.</span><span class="n">sql</span><span class="p">(),</span> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">source</span><span class="p">))</span>
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> </span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a>
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span> </span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="c1"># Find all columns that went into creating this one to list their lineage nodes.</span>
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="c1"># The table itself came from a more specific scope. Recurse into that one using the unaliased column name.</span> </span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="n">source_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">))</span>
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="n">to_node</span><span class="p">(</span> </span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a>
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="n">c</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">scope_name</span><span class="o">=</span><span class="n">table</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">node</span><span class="p">,</span> <span class="n">alias</span><span class="o">=</span><span class="n">aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> </span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="c1"># If the source is a UDTF find columns used in the UTDF to generate the table</span>
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="p">)</span> </span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">UDTF</span><span class="p">):</span>
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="n">source_columns</span> <span class="o">|=</span> <span class="nb">set</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">))</span>
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="c1"># The source is not a scope - we&#39;ve reached the end of the line. At this point, if a source is not found</span> </span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a>
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="c1"># it means this column&#39;s lineage is unknown. This can happen if the definition of a source used in a query</span> </span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">source_columns</span><span class="p">:</span>
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="c1"># is not passed into the `sources` map.</span> </span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">c</span><span class="o">.</span><span class="n">table</span>
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">source</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">Placeholder</span><span class="p">()</span> </span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="n">c</span><span class="o">.</span><span class="n">sql</span><span class="p">(),</span> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">source</span><span class="p">))</span> </span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a>
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> </span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span>
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="k">return</span> <span class="n">node</span> </span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="c1"># The table itself came from a more specific scope. Recurse into that one using the unaliased column name.</span>
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> </span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="n">to_node</span><span class="p">(</span>
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="k">return</span> <span class="n">to_node</span><span class="p">(</span><span class="n">column</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="p">)</span> </span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="n">c</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">scope_name</span><span class="o">=</span><span class="n">table</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">node</span><span class="p">,</span> <span class="n">alias</span><span class="o">=</span><span class="n">aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> </span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="p">)</span>
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> </span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a><span class="k">class</span> <span class="nc">LineageHTML</span><span class="p">:</span> </span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="c1"># The source is not a scope - we&#39;ve reached the end of the line. At this point, if a source is not found</span>
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;Node to HTML generator using vis.js.</span> </span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="c1"># it means this column&#39;s lineage is unknown. This can happen if the definition of a source used in a query</span>
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> </span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="c1"># is not passed into the `sources` map.</span>
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a><span class="sd"> https://visjs.github.io/vis-network/docs/network/</span> </span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">source</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">Placeholder</span><span class="p">()</span>
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="n">c</span><span class="o">.</span><span class="n">sql</span><span class="p">(),</span> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">source</span><span class="p">))</span>
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> </span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a>
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span> </span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="bp">self</span><span class="p">,</span> </span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a>
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="n">node</span><span class="p">:</span> <span class="n">Node</span><span class="p">,</span> </span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="k">return</span> <span class="n">to_node</span><span class="p">(</span><span class="n">column</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="p">)</span>
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a>
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="n">imports</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span> </span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a>
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span> </span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a><span class="k">class</span> <span class="nc">LineageHTML</span><span class="p">:</span>
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="p">):</span> </span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;Node to HTML generator using vis.js.</span>
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">node</span> <span class="o">=</span> <span class="n">node</span> </span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a>
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span> <span class="o">=</span> <span class="n">imports</span> </span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a><span class="sd"> https://visjs.github.io/vis-network/docs/network/</span>
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> </span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">options</span> <span class="o">=</span> <span class="p">{</span> </span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a>
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="s2">&quot;height&quot;</span><span class="p">:</span> <span class="s2">&quot;500px&quot;</span><span class="p">,</span> </span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span>
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="s2">&quot;width&quot;</span><span class="p">:</span> <span class="s2">&quot;100%&quot;</span><span class="p">,</span> </span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="bp">self</span><span class="p">,</span>
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a> <span class="s2">&quot;layout&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a> <span class="n">node</span><span class="p">:</span> <span class="n">Node</span><span class="p">,</span>
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a> <span class="s2">&quot;hierarchical&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a> <span class="s2">&quot;enabled&quot;</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span> </span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a> <span class="n">imports</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a> <span class="s2">&quot;nodeSpacing&quot;</span><span class="p">:</span> <span class="mi">200</span><span class="p">,</span> </span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a> <span class="s2">&quot;sortMethod&quot;</span><span class="p">:</span> <span class="s2">&quot;directed&quot;</span><span class="p">,</span> </span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a> <span class="p">):</span>
</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a> <span class="p">},</span> </span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">node</span> <span class="o">=</span> <span class="n">node</span>
</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a> <span class="p">},</span> </span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span> <span class="o">=</span> <span class="n">imports</span>
</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a> <span class="s2">&quot;interaction&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a>
</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="s2">&quot;dragNodes&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span> </span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a> <span class="s2">&quot;selectable&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span> </span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a> <span class="s2">&quot;height&quot;</span><span class="p">:</span> <span class="s2">&quot;500px&quot;</span><span class="p">,</span>
</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="p">},</span> </span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="s2">&quot;width&quot;</span><span class="p">:</span> <span class="s2">&quot;100%&quot;</span><span class="p">,</span>
</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a> <span class="s2">&quot;physics&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a> <span class="s2">&quot;layout&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a> <span class="s2">&quot;enabled&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span> </span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a> <span class="s2">&quot;hierarchical&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a> <span class="p">},</span> </span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a> <span class="s2">&quot;enabled&quot;</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a> <span class="s2">&quot;edges&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a> <span class="s2">&quot;nodeSpacing&quot;</span><span class="p">:</span> <span class="mi">200</span><span class="p">,</span>
</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a> <span class="s2">&quot;arrows&quot;</span><span class="p">:</span> <span class="s2">&quot;to&quot;</span><span class="p">,</span> </span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a> <span class="s2">&quot;sortMethod&quot;</span><span class="p">:</span> <span class="s2">&quot;directed&quot;</span><span class="p">,</span>
</span><span id="L-219"><a href="#L-219"><span class="linenos">219</span></a> <span class="p">},</span> </span><span id="L-219"><a href="#L-219"><span class="linenos">219</span></a> <span class="p">},</span>
</span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a> <span class="s2">&quot;nodes&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a> <span class="p">},</span>
</span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a> <span class="s2">&quot;font&quot;</span><span class="p">:</span> <span class="s2">&quot;20px monaco&quot;</span><span class="p">,</span> </span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a> <span class="s2">&quot;interaction&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a> <span class="s2">&quot;shape&quot;</span><span class="p">:</span> <span class="s2">&quot;box&quot;</span><span class="p">,</span> </span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a> <span class="s2">&quot;dragNodes&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="L-223"><a href="#L-223"><span class="linenos">223</span></a> <span class="s2">&quot;widthConstraint&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="L-223"><a href="#L-223"><span class="linenos">223</span></a> <span class="s2">&quot;selectable&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="L-224"><a href="#L-224"><span class="linenos">224</span></a> <span class="s2">&quot;maximum&quot;</span><span class="p">:</span> <span class="mi">300</span><span class="p">,</span> </span><span id="L-224"><a href="#L-224"><span class="linenos">224</span></a> <span class="p">},</span>
</span><span id="L-225"><a href="#L-225"><span class="linenos">225</span></a> <span class="p">},</span> </span><span id="L-225"><a href="#L-225"><span class="linenos">225</span></a> <span class="s2">&quot;physics&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="L-226"><a href="#L-226"><span class="linenos">226</span></a> <span class="p">},</span> </span><span id="L-226"><a href="#L-226"><span class="linenos">226</span></a> <span class="s2">&quot;enabled&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="L-227"><a href="#L-227"><span class="linenos">227</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">,</span> </span><span id="L-227"><a href="#L-227"><span class="linenos">227</span></a> <span class="p">},</span>
</span><span id="L-228"><a href="#L-228"><span class="linenos">228</span></a> <span class="p">}</span> </span><span id="L-228"><a href="#L-228"><span class="linenos">228</span></a> <span class="s2">&quot;edges&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="L-229"><a href="#L-229"><span class="linenos">229</span></a> </span><span id="L-229"><a href="#L-229"><span class="linenos">229</span></a> <span class="s2">&quot;arrows&quot;</span><span class="p">:</span> <span class="s2">&quot;to&quot;</span><span class="p">,</span>
</span><span id="L-230"><a href="#L-230"><span class="linenos">230</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span> </span><span id="L-230"><a href="#L-230"><span class="linenos">230</span></a> <span class="p">},</span>
</span><span id="L-231"><a href="#L-231"><span class="linenos">231</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span> <span class="o">=</span> <span class="p">[]</span> </span><span id="L-231"><a href="#L-231"><span class="linenos">231</span></a> <span class="s2">&quot;nodes&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="L-232"><a href="#L-232"><span class="linenos">232</span></a> </span><span id="L-232"><a href="#L-232"><span class="linenos">232</span></a> <span class="s2">&quot;font&quot;</span><span class="p">:</span> <span class="s2">&quot;20px monaco&quot;</span><span class="p">,</span>
</span><span id="L-233"><a href="#L-233"><span class="linenos">233</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span> </span><span id="L-233"><a href="#L-233"><span class="linenos">233</span></a> <span class="s2">&quot;shape&quot;</span><span class="p">:</span> <span class="s2">&quot;box&quot;</span><span class="p">,</span>
</span><span id="L-234"><a href="#L-234"><span class="linenos">234</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span> </span><span id="L-234"><a href="#L-234"><span class="linenos">234</span></a> <span class="s2">&quot;widthConstraint&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="L-235"><a href="#L-235"><span class="linenos">235</span></a> <span class="n">label</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">&quot;</span> </span><span id="L-235"><a href="#L-235"><span class="linenos">235</span></a> <span class="s2">&quot;maximum&quot;</span><span class="p">:</span> <span class="mi">300</span><span class="p">,</span>
</span><span id="L-236"><a href="#L-236"><span class="linenos">236</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;&lt;pre&gt;SELECT </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2"> FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">&lt;/pre&gt;&quot;</span> </span><span id="L-236"><a href="#L-236"><span class="linenos">236</span></a> <span class="p">},</span>
</span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">1</span> </span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a> <span class="p">},</span>
</span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">,</span>
</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a> <span class="n">label</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> </span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a> <span class="p">}</span>
</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">source</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span> </span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a>
</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a> <span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Tag</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">n</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">&quot;&lt;b&gt;&quot;</span><span class="p">,</span> <span class="n">postfix</span><span class="o">=</span><span class="s2">&quot;&lt;/b&gt;&quot;</span><span class="p">)</span> </span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a> <span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span> </span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span> <span class="o">=</span> <span class="p">[]</span>
</span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a> <span class="k">else</span> <span class="n">n</span><span class="p">,</span> </span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a>
</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> </span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a> <span class="p">)</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> </span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;&lt;pre&gt;</span><span class="si">{</span><span class="n">source</span><span class="si">}</span><span class="s2">&lt;/pre&gt;&quot;</span> </span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a> <span class="n">label</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">&quot;</span>
</span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">0</span> </span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;&lt;pre&gt;SELECT </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2"> FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">&lt;/pre&gt;&quot;</span>
</span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a> </span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">1</span>
</span><span id="L-249"><a href="#L-249"><span class="linenos">249</span></a> <span class="n">node_id</span> <span class="o">=</span> <span class="nb">id</span><span class="p">(</span><span class="n">node</span><span class="p">)</span> </span><span id="L-249"><a href="#L-249"><span class="linenos">249</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-250"><a href="#L-250"><span class="linenos">250</span></a> </span><span id="L-250"><a href="#L-250"><span class="linenos">250</span></a> <span class="n">label</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-251"><a href="#L-251"><span class="linenos">251</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="p">[</span><span class="n">node_id</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span> </span><span id="L-251"><a href="#L-251"><span class="linenos">251</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">source</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
</span><span id="L-252"><a href="#L-252"><span class="linenos">252</span></a> <span class="s2">&quot;id&quot;</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> </span><span id="L-252"><a href="#L-252"><span class="linenos">252</span></a> <span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Tag</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">n</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">&quot;&lt;b&gt;&quot;</span><span class="p">,</span> <span class="n">postfix</span><span class="o">=</span><span class="s2">&quot;&lt;/b&gt;&quot;</span><span class="p">)</span>
</span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a> <span class="s2">&quot;label&quot;</span><span class="p">:</span> <span class="n">label</span><span class="p">,</span> </span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a> <span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span>
</span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a> <span class="s2">&quot;title&quot;</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span> </span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a> <span class="k">else</span> <span class="n">n</span><span class="p">,</span>
</span><span id="L-255"><a href="#L-255"><span class="linenos">255</span></a> <span class="s2">&quot;group&quot;</span><span class="p">:</span> <span class="n">group</span><span class="p">,</span> </span><span id="L-255"><a href="#L-255"><span class="linenos">255</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a> <span class="p">}</span> </span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a> <span class="p">)</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a> </span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;&lt;pre&gt;</span><span class="si">{</span><span class="n">source</span><span class="si">}</span><span class="s2">&lt;/pre&gt;&quot;</span>
</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="p">:</span> </span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">0</span>
</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s2">&quot;from&quot;</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> <span class="s2">&quot;to&quot;</span><span class="p">:</span> <span class="nb">id</span><span class="p">(</span><span class="n">d</span><span class="p">)})</span> </span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a>
</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a> </span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a> <span class="n">node_id</span> <span class="o">=</span> <span class="nb">id</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a> <span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> </span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a>
</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="o">.</span><span class="n">values</span><span class="p">()))</span> </span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="p">[</span><span class="n">node_id</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a> <span class="n">edges</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="p">)</span> </span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a> <span class="s2">&quot;id&quot;</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span>
</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a> <span class="n">options</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">options</span><span class="p">)</span> </span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a> <span class="s2">&quot;label&quot;</span><span class="p">:</span> <span class="n">label</span><span class="p">,</span>
</span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a> <span class="n">imports</span> <span class="o">=</span> <span class="p">(</span> </span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a> <span class="s2">&quot;title&quot;</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span>
</span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;&lt;script type=&quot;text/javascript&quot; src=&quot;https://unpkg.com/vis-data@latest/peer/umd/vis-data.min.js&quot;&gt;&lt;/script&gt;</span> </span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a> <span class="s2">&quot;group&quot;</span><span class="p">:</span> <span class="n">group</span><span class="p">,</span>
</span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a><span class="sd"> &lt;script type=&quot;text/javascript&quot; src=&quot;https://unpkg.com/vis-network@latest/peer/umd/vis-network.min.js&quot;&gt;&lt;/script&gt;</span> </span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a> <span class="p">}</span>
</span><span id="L-268"><a href="#L-268"><span class="linenos">268</span></a><span class="sd"> &lt;link rel=&quot;stylesheet&quot; type=&quot;text/css&quot; href=&quot;https://unpkg.com/vis-network/styles/vis-network.min.css&quot; /&gt;&quot;&quot;&quot;</span> </span><span id="L-268"><a href="#L-268"><span class="linenos">268</span></a>
</span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span> </span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="p">:</span>
</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a> <span class="k">else</span> <span class="s2">&quot;&quot;</span> </span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s2">&quot;from&quot;</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> <span class="s2">&quot;to&quot;</span><span class="p">:</span> <span class="nb">id</span><span class="p">(</span><span class="n">d</span><span class="p">)})</span>
</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a> <span class="p">)</span> </span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a>
</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a> </span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a> <span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a> <span class="k">return</span> <span class="sa">f</span><span class="s2">&quot;&quot;&quot;&lt;div&gt;</span> </span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="o">.</span><span class="n">values</span><span class="p">()))</span>
</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a><span class="s2"> &lt;div id=&quot;sqlglot-lineage&quot;&gt;&lt;/div&gt;</span> </span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a> <span class="n">edges</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="p">)</span>
</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a><span class="s2"> </span><span class="si">{</span><span class="n">imports</span><span class="si">}</span> </span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a> <span class="n">options</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">options</span><span class="p">)</span>
</span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a><span class="s2"> &lt;script type=&quot;text/javascript&quot;&gt;</span> </span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a> <span class="n">imports</span> <span class="o">=</span> <span class="p">(</span>
</span><span id="L-277"><a href="#L-277"><span class="linenos">277</span></a><span class="s2"> var nodes = new vis.DataSet(</span><span class="si">{</span><span class="n">nodes</span><span class="si">}</span><span class="s2">)</span> </span><span id="L-277"><a href="#L-277"><span class="linenos">277</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;&lt;script type=&quot;text/javascript&quot; src=&quot;https://unpkg.com/vis-data@latest/peer/umd/vis-data.min.js&quot;&gt;&lt;/script&gt;</span>
</span><span id="L-278"><a href="#L-278"><span class="linenos">278</span></a><span class="s2"> nodes.forEach(row =&gt; row[&quot;title&quot;] = new DOMParser().parseFromString(row[&quot;title&quot;], &quot;text/html&quot;).body.childNodes[0])</span> </span><span id="L-278"><a href="#L-278"><span class="linenos">278</span></a><span class="sd"> &lt;script type=&quot;text/javascript&quot; src=&quot;https://unpkg.com/vis-network@latest/peer/umd/vis-network.min.js&quot;&gt;&lt;/script&gt;</span>
</span><span id="L-279"><a href="#L-279"><span class="linenos">279</span></a> </span><span id="L-279"><a href="#L-279"><span class="linenos">279</span></a><span class="sd"> &lt;link rel=&quot;stylesheet&quot; type=&quot;text/css&quot; href=&quot;https://unpkg.com/vis-network/styles/vis-network.min.css&quot; /&gt;&quot;&quot;&quot;</span>
</span><span id="L-280"><a href="#L-280"><span class="linenos">280</span></a><span class="s2"> new vis.Network(</span> </span><span id="L-280"><a href="#L-280"><span class="linenos">280</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span>
</span><span id="L-281"><a href="#L-281"><span class="linenos">281</span></a><span class="s2"> document.getElementById(&quot;sqlglot-lineage&quot;),</span> </span><span id="L-281"><a href="#L-281"><span class="linenos">281</span></a> <span class="k">else</span> <span class="s2">&quot;&quot;</span>
</span><span id="L-282"><a href="#L-282"><span class="linenos">282</span></a><span class="s2"> </span><span class="se">{{</span> </span><span id="L-282"><a href="#L-282"><span class="linenos">282</span></a> <span class="p">)</span>
</span><span id="L-283"><a href="#L-283"><span class="linenos">283</span></a><span class="s2"> nodes: nodes,</span> </span><span id="L-283"><a href="#L-283"><span class="linenos">283</span></a>
</span><span id="L-284"><a href="#L-284"><span class="linenos">284</span></a><span class="s2"> edges: new vis.DataSet(</span><span class="si">{</span><span class="n">edges</span><span class="si">}</span><span class="s2">)</span> </span><span id="L-284"><a href="#L-284"><span class="linenos">284</span></a> <span class="k">return</span> <span class="sa">f</span><span class="s2">&quot;&quot;&quot;&lt;div&gt;</span>
</span><span id="L-285"><a href="#L-285"><span class="linenos">285</span></a><span class="s2"> </span><span class="se">}}</span><span class="s2">,</span> </span><span id="L-285"><a href="#L-285"><span class="linenos">285</span></a><span class="s2"> &lt;div id=&quot;sqlglot-lineage&quot;&gt;&lt;/div&gt;</span>
</span><span id="L-286"><a href="#L-286"><span class="linenos">286</span></a><span class="s2"> </span><span class="si">{</span><span class="n">options</span><span class="si">}</span><span class="s2">,</span> </span><span id="L-286"><a href="#L-286"><span class="linenos">286</span></a><span class="s2"> </span><span class="si">{</span><span class="n">imports</span><span class="si">}</span>
</span><span id="L-287"><a href="#L-287"><span class="linenos">287</span></a><span class="s2"> )</span> </span><span id="L-287"><a href="#L-287"><span class="linenos">287</span></a><span class="s2"> &lt;script type=&quot;text/javascript&quot;&gt;</span>
</span><span id="L-288"><a href="#L-288"><span class="linenos">288</span></a><span class="s2"> &lt;/script&gt;</span> </span><span id="L-288"><a href="#L-288"><span class="linenos">288</span></a><span class="s2"> var nodes = new vis.DataSet(</span><span class="si">{</span><span class="n">nodes</span><span class="si">}</span><span class="s2">)</span>
</span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a><span class="s2">&lt;/div&gt;&quot;&quot;&quot;</span> </span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a><span class="s2"> nodes.forEach(row =&gt; row[&quot;title&quot;] = new DOMParser().parseFromString(row[&quot;title&quot;], &quot;text/html&quot;).body.childNodes[0])</span>
</span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a> </span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a>
</span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a> <span class="k">def</span> <span class="nf">_repr_html_</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span> </span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a><span class="s2"> new vis.Network(</span>
</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="fm">__str__</span><span class="p">()</span> </span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a><span class="s2"> document.getElementById(&quot;sqlglot-lineage&quot;),</span>
</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a><span class="s2"> </span><span class="se">{{</span>
</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a><span class="s2"> nodes: nodes,</span>
</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a><span class="s2"> edges: new vis.DataSet(</span><span class="si">{</span><span class="n">edges</span><span class="si">}</span><span class="s2">)</span>
</span><span id="L-296"><a href="#L-296"><span class="linenos">296</span></a><span class="s2"> </span><span class="se">}}</span><span class="s2">,</span>
</span><span id="L-297"><a href="#L-297"><span class="linenos">297</span></a><span class="s2"> </span><span class="si">{</span><span class="n">options</span><span class="si">}</span><span class="s2">,</span>
</span><span id="L-298"><a href="#L-298"><span class="linenos">298</span></a><span class="s2"> )</span>
</span><span id="L-299"><a href="#L-299"><span class="linenos">299</span></a><span class="s2"> &lt;/script&gt;</span>
</span><span id="L-300"><a href="#L-300"><span class="linenos">300</span></a><span class="s2">&lt;/div&gt;&quot;&quot;&quot;</span>
</span><span id="L-301"><a href="#L-301"><span class="linenos">301</span></a>
</span><span id="L-302"><a href="#L-302"><span class="linenos">302</span></a> <span class="k">def</span> <span class="nf">_repr_html_</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
</span><span id="L-303"><a href="#L-303"><span class="linenos">303</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="fm">__str__</span><span class="p">()</span>
</span></pre></div> </span></pre></div>
@ -645,73 +656,84 @@
</span><span id="lineage-113"><a href="#lineage-113"><span class="linenos">113</span></a> <span class="n">column</span> </span><span id="lineage-113"><a href="#lineage-113"><span class="linenos">113</span></a> <span class="n">column</span>
</span><span id="lineage-114"><a href="#lineage-114"><span class="linenos">114</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">int</span><span class="p">)</span> </span><span id="lineage-114"><a href="#lineage-114"><span class="linenos">114</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">int</span><span class="p">)</span>
</span><span id="lineage-115"><a href="#lineage-115"><span class="linenos">115</span></a> <span class="k">else</span> <span class="nb">next</span><span class="p">(</span> </span><span id="lineage-115"><a href="#lineage-115"><span class="linenos">115</span></a> <span class="k">else</span> <span class="nb">next</span><span class="p">(</span>
</span><span id="lineage-116"><a href="#lineage-116"><span class="linenos">116</span></a> <span class="n">i</span> </span><span id="lineage-116"><a href="#lineage-116"><span class="linenos">116</span></a> <span class="p">(</span>
</span><span id="lineage-117"><a href="#lineage-117"><span class="linenos">117</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">select</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span> </span><span id="lineage-117"><a href="#lineage-117"><span class="linenos">117</span></a> <span class="n">i</span>
</span><span id="lineage-118"><a href="#lineage-118"><span class="linenos">118</span></a> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">column</span> </span><span id="lineage-118"><a href="#lineage-118"><span class="linenos">118</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">select</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span>
</span><span id="lineage-119"><a href="#lineage-119"><span class="linenos">119</span></a> <span class="p">)</span> </span><span id="lineage-119"><a href="#lineage-119"><span class="linenos">119</span></a> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">column</span> <span class="ow">or</span> <span class="n">select</span><span class="o">.</span><span class="n">is_star</span>
</span><span id="lineage-120"><a href="#lineage-120"><span class="linenos">120</span></a> <span class="p">)</span> </span><span id="lineage-120"><a href="#lineage-120"><span class="linenos">120</span></a> <span class="p">),</span>
</span><span id="lineage-121"><a href="#lineage-121"><span class="linenos">121</span></a> </span><span id="lineage-121"><a href="#lineage-121"><span class="linenos">121</span></a> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="c1"># mypy will not allow a None here, but a negative index should never be returned</span>
</span><span id="lineage-122"><a href="#lineage-122"><span class="linenos">122</span></a> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">union_scopes</span><span class="p">:</span> </span><span id="lineage-122"><a href="#lineage-122"><span class="linenos">122</span></a> <span class="p">)</span>
</span><span id="lineage-123"><a href="#lineage-123"><span class="linenos">123</span></a> <span class="n">to_node</span><span class="p">(</span><span class="n">index</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">s</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">upstream</span><span class="p">)</span> </span><span id="lineage-123"><a href="#lineage-123"><span class="linenos">123</span></a> <span class="p">)</span>
</span><span id="lineage-124"><a href="#lineage-124"><span class="linenos">124</span></a> </span><span id="lineage-124"><a href="#lineage-124"><span class="linenos">124</span></a>
</span><span id="lineage-125"><a href="#lineage-125"><span class="linenos">125</span></a> <span class="k">return</span> <span class="n">upstream</span> </span><span id="lineage-125"><a href="#lineage-125"><span class="linenos">125</span></a> <span class="k">if</span> <span class="n">index</span> <span class="o">==</span> <span class="o">-</span><span class="mi">1</span><span class="p">:</span>
</span><span id="lineage-126"><a href="#lineage-126"><span class="linenos">126</span></a> </span><span id="lineage-126"><a href="#lineage-126"><span class="linenos">126</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;Could not find </span><span class="si">{</span><span class="n">column</span><span class="si">}</span><span class="s2"> in </span><span class="si">{</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">)</span>
</span><span id="lineage-127"><a href="#lineage-127"><span class="linenos">127</span></a> <span class="n">subquery</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">unalias</span><span class="p">()</span> </span><span id="lineage-127"><a href="#lineage-127"><span class="linenos">127</span></a>
</span><span id="lineage-128"><a href="#lineage-128"><span class="linenos">128</span></a> </span><span id="lineage-128"><a href="#lineage-128"><span class="linenos">128</span></a> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">union_scopes</span><span class="p">:</span>
</span><span id="lineage-129"><a href="#lineage-129"><span class="linenos">129</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">subquery</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span> </span><span id="lineage-129"><a href="#lineage-129"><span class="linenos">129</span></a> <span class="n">to_node</span><span class="p">(</span><span class="n">index</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">s</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">upstream</span><span class="p">)</span>
</span><span id="lineage-130"><a href="#lineage-130"><span class="linenos">130</span></a> <span class="n">upstream</span> <span class="o">=</span> <span class="n">upstream</span> <span class="ow">or</span> <span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&quot;SUBQUERY&quot;</span><span class="p">,</span> <span class="n">source</span><span class="o">=</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">select</span><span class="p">)</span> </span><span id="lineage-130"><a href="#lineage-130"><span class="linenos">130</span></a>
</span><span id="lineage-131"><a href="#lineage-131"><span class="linenos">131</span></a> <span class="n">scope</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">Scope</span><span class="p">,</span> <span class="n">build_scope</span><span class="p">(</span><span class="n">subquery</span><span class="o">.</span><span class="n">unnest</span><span class="p">()))</span> </span><span id="lineage-131"><a href="#lineage-131"><span class="linenos">131</span></a> <span class="k">return</span> <span class="n">upstream</span>
</span><span id="lineage-132"><a href="#lineage-132"><span class="linenos">132</span></a> </span><span id="lineage-132"><a href="#lineage-132"><span class="linenos">132</span></a>
</span><span id="lineage-133"><a href="#lineage-133"><span class="linenos">133</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">subquery</span><span class="o">.</span><span class="n">named_selects</span><span class="p">:</span> </span><span id="lineage-133"><a href="#lineage-133"><span class="linenos">133</span></a> <span class="n">subquery</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">unalias</span><span class="p">()</span>
</span><span id="lineage-134"><a href="#lineage-134"><span class="linenos">134</span></a> <span class="n">to_node</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">scope</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">upstream</span><span class="p">)</span> </span><span id="lineage-134"><a href="#lineage-134"><span class="linenos">134</span></a>
</span><span id="lineage-135"><a href="#lineage-135"><span class="linenos">135</span></a> </span><span id="lineage-135"><a href="#lineage-135"><span class="linenos">135</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">subquery</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
</span><span id="lineage-136"><a href="#lineage-136"><span class="linenos">136</span></a> <span class="k">return</span> <span class="n">upstream</span> </span><span id="lineage-136"><a href="#lineage-136"><span class="linenos">136</span></a> <span class="n">upstream</span> <span class="o">=</span> <span class="n">upstream</span> <span class="ow">or</span> <span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">&quot;SUBQUERY&quot;</span><span class="p">,</span> <span class="n">source</span><span class="o">=</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">select</span><span class="p">)</span>
</span><span id="lineage-137"><a href="#lineage-137"><span class="linenos">137</span></a> </span><span id="lineage-137"><a href="#lineage-137"><span class="linenos">137</span></a> <span class="n">scope</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">Scope</span><span class="p">,</span> <span class="n">build_scope</span><span class="p">(</span><span class="n">subquery</span><span class="o">.</span><span class="n">unnest</span><span class="p">()))</span>
</span><span id="lineage-138"><a href="#lineage-138"><span class="linenos">138</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span> </span><span id="lineage-138"><a href="#lineage-138"><span class="linenos">138</span></a>
</span><span id="lineage-139"><a href="#lineage-139"><span class="linenos">139</span></a> <span class="c1"># For better ergonomics in our node labels, replace the full select with</span> </span><span id="lineage-139"><a href="#lineage-139"><span class="linenos">139</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">subquery</span><span class="o">.</span><span class="n">named_selects</span><span class="p">:</span>
</span><span id="lineage-140"><a href="#lineage-140"><span class="linenos">140</span></a> <span class="c1"># a version that has only the column we care about.</span> </span><span id="lineage-140"><a href="#lineage-140"><span class="linenos">140</span></a> <span class="n">to_node</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">scope</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">upstream</span><span class="p">)</span>
</span><span id="lineage-141"><a href="#lineage-141"><span class="linenos">141</span></a> <span class="c1"># &quot;x&quot;, SELECT x, y FROM foo</span> </span><span id="lineage-141"><a href="#lineage-141"><span class="linenos">141</span></a>
</span><span id="lineage-142"><a href="#lineage-142"><span class="linenos">142</span></a> <span class="c1"># =&gt; &quot;x&quot;, SELECT x FROM foo</span> </span><span id="lineage-142"><a href="#lineage-142"><span class="linenos">142</span></a> <span class="k">return</span> <span class="n">upstream</span>
</span><span id="lineage-143"><a href="#lineage-143"><span class="linenos">143</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">append</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span> </span><span id="lineage-143"><a href="#lineage-143"><span class="linenos">143</span></a>
</span><span id="lineage-144"><a href="#lineage-144"><span class="linenos">144</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="lineage-144"><a href="#lineage-144"><span class="linenos">144</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
</span><span id="lineage-145"><a href="#lineage-145"><span class="linenos">145</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span> </span><span id="lineage-145"><a href="#lineage-145"><span class="linenos">145</span></a> <span class="c1"># For better ergonomics in our node labels, replace the full select with</span>
</span><span id="lineage-146"><a href="#lineage-146"><span class="linenos">146</span></a> </span><span id="lineage-146"><a href="#lineage-146"><span class="linenos">146</span></a> <span class="c1"># a version that has only the column we care about.</span>
</span><span id="lineage-147"><a href="#lineage-147"><span class="linenos">147</span></a> <span class="c1"># Create the node for this step in the lineage chain, and attach it to the previous one.</span> </span><span id="lineage-147"><a href="#lineage-147"><span class="linenos">147</span></a> <span class="c1"># &quot;x&quot;, SELECT x, y FROM foo</span>
</span><span id="lineage-148"><a href="#lineage-148"><span class="linenos">148</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">Node</span><span class="p">(</span> </span><span id="lineage-148"><a href="#lineage-148"><span class="linenos">148</span></a> <span class="c1"># =&gt; &quot;x&quot;, SELECT x FROM foo</span>
</span><span id="lineage-149"><a href="#lineage-149"><span class="linenos">149</span></a> <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;</span><span class="si">{</span><span class="n">scope_name</span><span class="si">}</span><span class="s2">.</span><span class="si">{</span><span class="n">column</span><span class="si">}</span><span class="s2">&quot;</span> <span class="k">if</span> <span class="n">scope_name</span> <span class="k">else</span> <span class="nb">str</span><span class="p">(</span><span class="n">column</span><span class="p">),</span> </span><span id="lineage-149"><a href="#lineage-149"><span class="linenos">149</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">append</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
</span><span id="lineage-150"><a href="#lineage-150"><span class="linenos">150</span></a> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> </span><span id="lineage-150"><a href="#lineage-150"><span class="linenos">150</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="lineage-151"><a href="#lineage-151"><span class="linenos">151</span></a> <span class="n">expression</span><span class="o">=</span><span class="n">select</span><span class="p">,</span> </span><span id="lineage-151"><a href="#lineage-151"><span class="linenos">151</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span>
</span><span id="lineage-152"><a href="#lineage-152"><span class="linenos">152</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">alias</span> <span class="ow">or</span> <span class="s2">&quot;&quot;</span><span class="p">,</span> </span><span id="lineage-152"><a href="#lineage-152"><span class="linenos">152</span></a>
</span><span id="lineage-153"><a href="#lineage-153"><span class="linenos">153</span></a> <span class="p">)</span> </span><span id="lineage-153"><a href="#lineage-153"><span class="linenos">153</span></a> <span class="c1"># Create the node for this step in the lineage chain, and attach it to the previous one.</span>
</span><span id="lineage-154"><a href="#lineage-154"><span class="linenos">154</span></a> <span class="k">if</span> <span class="n">upstream</span><span class="p">:</span> </span><span id="lineage-154"><a href="#lineage-154"><span class="linenos">154</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">Node</span><span class="p">(</span>
</span><span id="lineage-155"><a href="#lineage-155"><span class="linenos">155</span></a> <span class="n">upstream</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">node</span><span class="p">)</span> </span><span id="lineage-155"><a href="#lineage-155"><span class="linenos">155</span></a> <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;</span><span class="si">{</span><span class="n">scope_name</span><span class="si">}</span><span class="s2">.</span><span class="si">{</span><span class="n">column</span><span class="si">}</span><span class="s2">&quot;</span> <span class="k">if</span> <span class="n">scope_name</span> <span class="k">else</span> <span class="nb">str</span><span class="p">(</span><span class="n">column</span><span class="p">),</span>
</span><span id="lineage-156"><a href="#lineage-156"><span class="linenos">156</span></a> </span><span id="lineage-156"><a href="#lineage-156"><span class="linenos">156</span></a> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span>
</span><span id="lineage-157"><a href="#lineage-157"><span class="linenos">157</span></a> <span class="c1"># Find all columns that went into creating this one to list their lineage nodes.</span> </span><span id="lineage-157"><a href="#lineage-157"><span class="linenos">157</span></a> <span class="n">expression</span><span class="o">=</span><span class="n">select</span><span class="p">,</span>
</span><span id="lineage-158"><a href="#lineage-158"><span class="linenos">158</span></a> <span class="n">source_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">))</span> </span><span id="lineage-158"><a href="#lineage-158"><span class="linenos">158</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">alias</span> <span class="ow">or</span> <span class="s2">&quot;&quot;</span><span class="p">,</span>
</span><span id="lineage-159"><a href="#lineage-159"><span class="linenos">159</span></a> </span><span id="lineage-159"><a href="#lineage-159"><span class="linenos">159</span></a> <span class="p">)</span>
</span><span id="lineage-160"><a href="#lineage-160"><span class="linenos">160</span></a> <span class="c1"># If the source is a UDTF find columns used in the UTDF to generate the table</span> </span><span id="lineage-160"><a href="#lineage-160"><span class="linenos">160</span></a> <span class="k">if</span> <span class="n">upstream</span><span class="p">:</span>
</span><span id="lineage-161"><a href="#lineage-161"><span class="linenos">161</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">UDTF</span><span class="p">):</span> </span><span id="lineage-161"><a href="#lineage-161"><span class="linenos">161</span></a> <span class="n">upstream</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="lineage-162"><a href="#lineage-162"><span class="linenos">162</span></a> <span class="n">source_columns</span> <span class="o">|=</span> <span class="nb">set</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">))</span> </span><span id="lineage-162"><a href="#lineage-162"><span class="linenos">162</span></a>
</span><span id="lineage-163"><a href="#lineage-163"><span class="linenos">163</span></a> </span><span id="lineage-163"><a href="#lineage-163"><span class="linenos">163</span></a> <span class="c1"># if the select is a star add all scope sources as downstreams</span>
</span><span id="lineage-164"><a href="#lineage-164"><span class="linenos">164</span></a> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">source_columns</span><span class="p">:</span> </span><span id="lineage-164"><a href="#lineage-164"><span class="linenos">164</span></a> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">is_star</span><span class="p">:</span>
</span><span id="lineage-165"><a href="#lineage-165"><span class="linenos">165</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">c</span><span class="o">.</span><span class="n">table</span> </span><span id="lineage-165"><a href="#lineage-165"><span class="linenos">165</span></a> <span class="k">for</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
</span><span id="lineage-166"><a href="#lineage-166"><span class="linenos">166</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> </span><span id="lineage-166"><a href="#lineage-166"><span class="linenos">166</span></a> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="n">select</span><span class="o">.</span><span class="n">sql</span><span class="p">(),</span> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">source</span><span class="p">))</span>
</span><span id="lineage-167"><a href="#lineage-167"><span class="linenos">167</span></a> </span><span id="lineage-167"><a href="#lineage-167"><span class="linenos">167</span></a>
</span><span id="lineage-168"><a href="#lineage-168"><span class="linenos">168</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span> </span><span id="lineage-168"><a href="#lineage-168"><span class="linenos">168</span></a> <span class="c1"># Find all columns that went into creating this one to list their lineage nodes.</span>
</span><span id="lineage-169"><a href="#lineage-169"><span class="linenos">169</span></a> <span class="c1"># The table itself came from a more specific scope. Recurse into that one using the unaliased column name.</span> </span><span id="lineage-169"><a href="#lineage-169"><span class="linenos">169</span></a> <span class="n">source_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">))</span>
</span><span id="lineage-170"><a href="#lineage-170"><span class="linenos">170</span></a> <span class="n">to_node</span><span class="p">(</span> </span><span id="lineage-170"><a href="#lineage-170"><span class="linenos">170</span></a>
</span><span id="lineage-171"><a href="#lineage-171"><span class="linenos">171</span></a> <span class="n">c</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">scope_name</span><span class="o">=</span><span class="n">table</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">node</span><span class="p">,</span> <span class="n">alias</span><span class="o">=</span><span class="n">aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> </span><span id="lineage-171"><a href="#lineage-171"><span class="linenos">171</span></a> <span class="c1"># If the source is a UDTF find columns used in the UTDF to generate the table</span>
</span><span id="lineage-172"><a href="#lineage-172"><span class="linenos">172</span></a> <span class="p">)</span> </span><span id="lineage-172"><a href="#lineage-172"><span class="linenos">172</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">UDTF</span><span class="p">):</span>
</span><span id="lineage-173"><a href="#lineage-173"><span class="linenos">173</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="lineage-173"><a href="#lineage-173"><span class="linenos">173</span></a> <span class="n">source_columns</span> <span class="o">|=</span> <span class="nb">set</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">))</span>
</span><span id="lineage-174"><a href="#lineage-174"><span class="linenos">174</span></a> <span class="c1"># The source is not a scope - we&#39;ve reached the end of the line. At this point, if a source is not found</span> </span><span id="lineage-174"><a href="#lineage-174"><span class="linenos">174</span></a>
</span><span id="lineage-175"><a href="#lineage-175"><span class="linenos">175</span></a> <span class="c1"># it means this column&#39;s lineage is unknown. This can happen if the definition of a source used in a query</span> </span><span id="lineage-175"><a href="#lineage-175"><span class="linenos">175</span></a> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">source_columns</span><span class="p">:</span>
</span><span id="lineage-176"><a href="#lineage-176"><span class="linenos">176</span></a> <span class="c1"># is not passed into the `sources` map.</span> </span><span id="lineage-176"><a href="#lineage-176"><span class="linenos">176</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">c</span><span class="o">.</span><span class="n">table</span>
</span><span id="lineage-177"><a href="#lineage-177"><span class="linenos">177</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">source</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">Placeholder</span><span class="p">()</span> </span><span id="lineage-177"><a href="#lineage-177"><span class="linenos">177</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
</span><span id="lineage-178"><a href="#lineage-178"><span class="linenos">178</span></a> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="n">c</span><span class="o">.</span><span class="n">sql</span><span class="p">(),</span> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">source</span><span class="p">))</span> </span><span id="lineage-178"><a href="#lineage-178"><span class="linenos">178</span></a>
</span><span id="lineage-179"><a href="#lineage-179"><span class="linenos">179</span></a> </span><span id="lineage-179"><a href="#lineage-179"><span class="linenos">179</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span>
</span><span id="lineage-180"><a href="#lineage-180"><span class="linenos">180</span></a> <span class="k">return</span> <span class="n">node</span> </span><span id="lineage-180"><a href="#lineage-180"><span class="linenos">180</span></a> <span class="c1"># The table itself came from a more specific scope. Recurse into that one using the unaliased column name.</span>
</span><span id="lineage-181"><a href="#lineage-181"><span class="linenos">181</span></a> </span><span id="lineage-181"><a href="#lineage-181"><span class="linenos">181</span></a> <span class="n">to_node</span><span class="p">(</span>
</span><span id="lineage-182"><a href="#lineage-182"><span class="linenos">182</span></a> <span class="k">return</span> <span class="n">to_node</span><span class="p">(</span><span class="n">column</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="p">)</span> </span><span id="lineage-182"><a href="#lineage-182"><span class="linenos">182</span></a> <span class="n">c</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">scope_name</span><span class="o">=</span><span class="n">table</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">node</span><span class="p">,</span> <span class="n">alias</span><span class="o">=</span><span class="n">aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
</span><span id="lineage-183"><a href="#lineage-183"><span class="linenos">183</span></a> <span class="p">)</span>
</span><span id="lineage-184"><a href="#lineage-184"><span class="linenos">184</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="lineage-185"><a href="#lineage-185"><span class="linenos">185</span></a> <span class="c1"># The source is not a scope - we&#39;ve reached the end of the line. At this point, if a source is not found</span>
</span><span id="lineage-186"><a href="#lineage-186"><span class="linenos">186</span></a> <span class="c1"># it means this column&#39;s lineage is unknown. This can happen if the definition of a source used in a query</span>
</span><span id="lineage-187"><a href="#lineage-187"><span class="linenos">187</span></a> <span class="c1"># is not passed into the `sources` map.</span>
</span><span id="lineage-188"><a href="#lineage-188"><span class="linenos">188</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">source</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">Placeholder</span><span class="p">()</span>
</span><span id="lineage-189"><a href="#lineage-189"><span class="linenos">189</span></a> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="n">c</span><span class="o">.</span><span class="n">sql</span><span class="p">(),</span> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">source</span><span class="p">))</span>
</span><span id="lineage-190"><a href="#lineage-190"><span class="linenos">190</span></a>
</span><span id="lineage-191"><a href="#lineage-191"><span class="linenos">191</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="lineage-192"><a href="#lineage-192"><span class="linenos">192</span></a>
</span><span id="lineage-193"><a href="#lineage-193"><span class="linenos">193</span></a> <span class="k">return</span> <span class="n">to_node</span><span class="p">(</span><span class="n">column</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="p">)</span>
</span></pre></div> </span></pre></div>
@ -748,115 +770,115 @@
</div> </div>
<a class="headerlink" href="#LineageHTML"></a> <a class="headerlink" href="#LineageHTML"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="LineageHTML-185"><a href="#LineageHTML-185"><span class="linenos">185</span></a><span class="k">class</span> <span class="nc">LineageHTML</span><span class="p">:</span> <div class="pdoc-code codehilite"><pre><span></span><span id="LineageHTML-196"><a href="#LineageHTML-196"><span class="linenos">196</span></a><span class="k">class</span> <span class="nc">LineageHTML</span><span class="p">:</span>
</span><span id="LineageHTML-186"><a href="#LineageHTML-186"><span class="linenos">186</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;Node to HTML generator using vis.js.</span> </span><span id="LineageHTML-197"><a href="#LineageHTML-197"><span class="linenos">197</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;Node to HTML generator using vis.js.</span>
</span><span id="LineageHTML-187"><a href="#LineageHTML-187"><span class="linenos">187</span></a> </span><span id="LineageHTML-198"><a href="#LineageHTML-198"><span class="linenos">198</span></a>
</span><span id="LineageHTML-188"><a href="#LineageHTML-188"><span class="linenos">188</span></a><span class="sd"> https://visjs.github.io/vis-network/docs/network/</span> </span><span id="LineageHTML-199"><a href="#LineageHTML-199"><span class="linenos">199</span></a><span class="sd"> https://visjs.github.io/vis-network/docs/network/</span>
</span><span id="LineageHTML-189"><a href="#LineageHTML-189"><span class="linenos">189</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="LineageHTML-200"><a href="#LineageHTML-200"><span class="linenos">200</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="LineageHTML-190"><a href="#LineageHTML-190"><span class="linenos">190</span></a> </span><span id="LineageHTML-201"><a href="#LineageHTML-201"><span class="linenos">201</span></a>
</span><span id="LineageHTML-191"><a href="#LineageHTML-191"><span class="linenos">191</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span> </span><span id="LineageHTML-202"><a href="#LineageHTML-202"><span class="linenos">202</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span>
</span><span id="LineageHTML-192"><a href="#LineageHTML-192"><span class="linenos">192</span></a> <span class="bp">self</span><span class="p">,</span> </span><span id="LineageHTML-203"><a href="#LineageHTML-203"><span class="linenos">203</span></a> <span class="bp">self</span><span class="p">,</span>
</span><span id="LineageHTML-193"><a href="#LineageHTML-193"><span class="linenos">193</span></a> <span class="n">node</span><span class="p">:</span> <span class="n">Node</span><span class="p">,</span> </span><span id="LineageHTML-204"><a href="#LineageHTML-204"><span class="linenos">204</span></a> <span class="n">node</span><span class="p">:</span> <span class="n">Node</span><span class="p">,</span>
</span><span id="LineageHTML-194"><a href="#LineageHTML-194"><span class="linenos">194</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="LineageHTML-205"><a href="#LineageHTML-205"><span class="linenos">205</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="LineageHTML-195"><a href="#LineageHTML-195"><span class="linenos">195</span></a> <span class="n">imports</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span> </span><span id="LineageHTML-206"><a href="#LineageHTML-206"><span class="linenos">206</span></a> <span class="n">imports</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="LineageHTML-196"><a href="#LineageHTML-196"><span class="linenos">196</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span> </span><span id="LineageHTML-207"><a href="#LineageHTML-207"><span class="linenos">207</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
</span><span id="LineageHTML-197"><a href="#LineageHTML-197"><span class="linenos">197</span></a> <span class="p">):</span> </span><span id="LineageHTML-208"><a href="#LineageHTML-208"><span class="linenos">208</span></a> <span class="p">):</span>
</span><span id="LineageHTML-198"><a href="#LineageHTML-198"><span class="linenos">198</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">node</span> <span class="o">=</span> <span class="n">node</span> </span><span id="LineageHTML-209"><a href="#LineageHTML-209"><span class="linenos">209</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">node</span> <span class="o">=</span> <span class="n">node</span>
</span><span id="LineageHTML-199"><a href="#LineageHTML-199"><span class="linenos">199</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span> <span class="o">=</span> <span class="n">imports</span> </span><span id="LineageHTML-210"><a href="#LineageHTML-210"><span class="linenos">210</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span> <span class="o">=</span> <span class="n">imports</span>
</span><span id="LineageHTML-200"><a href="#LineageHTML-200"><span class="linenos">200</span></a> </span><span id="LineageHTML-211"><a href="#LineageHTML-211"><span class="linenos">211</span></a>
</span><span id="LineageHTML-201"><a href="#LineageHTML-201"><span class="linenos">201</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">options</span> <span class="o">=</span> <span class="p">{</span> </span><span id="LineageHTML-212"><a href="#LineageHTML-212"><span class="linenos">212</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="LineageHTML-202"><a href="#LineageHTML-202"><span class="linenos">202</span></a> <span class="s2">&quot;height&quot;</span><span class="p">:</span> <span class="s2">&quot;500px&quot;</span><span class="p">,</span> </span><span id="LineageHTML-213"><a href="#LineageHTML-213"><span class="linenos">213</span></a> <span class="s2">&quot;height&quot;</span><span class="p">:</span> <span class="s2">&quot;500px&quot;</span><span class="p">,</span>
</span><span id="LineageHTML-203"><a href="#LineageHTML-203"><span class="linenos">203</span></a> <span class="s2">&quot;width&quot;</span><span class="p">:</span> <span class="s2">&quot;100%&quot;</span><span class="p">,</span> </span><span id="LineageHTML-214"><a href="#LineageHTML-214"><span class="linenos">214</span></a> <span class="s2">&quot;width&quot;</span><span class="p">:</span> <span class="s2">&quot;100%&quot;</span><span class="p">,</span>
</span><span id="LineageHTML-204"><a href="#LineageHTML-204"><span class="linenos">204</span></a> <span class="s2">&quot;layout&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="LineageHTML-215"><a href="#LineageHTML-215"><span class="linenos">215</span></a> <span class="s2">&quot;layout&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML-205"><a href="#LineageHTML-205"><span class="linenos">205</span></a> <span class="s2">&quot;hierarchical&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="LineageHTML-216"><a href="#LineageHTML-216"><span class="linenos">216</span></a> <span class="s2">&quot;hierarchical&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML-206"><a href="#LineageHTML-206"><span class="linenos">206</span></a> <span class="s2">&quot;enabled&quot;</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span> </span><span id="LineageHTML-217"><a href="#LineageHTML-217"><span class="linenos">217</span></a> <span class="s2">&quot;enabled&quot;</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="LineageHTML-207"><a href="#LineageHTML-207"><span class="linenos">207</span></a> <span class="s2">&quot;nodeSpacing&quot;</span><span class="p">:</span> <span class="mi">200</span><span class="p">,</span> </span><span id="LineageHTML-218"><a href="#LineageHTML-218"><span class="linenos">218</span></a> <span class="s2">&quot;nodeSpacing&quot;</span><span class="p">:</span> <span class="mi">200</span><span class="p">,</span>
</span><span id="LineageHTML-208"><a href="#LineageHTML-208"><span class="linenos">208</span></a> <span class="s2">&quot;sortMethod&quot;</span><span class="p">:</span> <span class="s2">&quot;directed&quot;</span><span class="p">,</span> </span><span id="LineageHTML-219"><a href="#LineageHTML-219"><span class="linenos">219</span></a> <span class="s2">&quot;sortMethod&quot;</span><span class="p">:</span> <span class="s2">&quot;directed&quot;</span><span class="p">,</span>
</span><span id="LineageHTML-209"><a href="#LineageHTML-209"><span class="linenos">209</span></a> <span class="p">},</span>
</span><span id="LineageHTML-210"><a href="#LineageHTML-210"><span class="linenos">210</span></a> <span class="p">},</span>
</span><span id="LineageHTML-211"><a href="#LineageHTML-211"><span class="linenos">211</span></a> <span class="s2">&quot;interaction&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML-212"><a href="#LineageHTML-212"><span class="linenos">212</span></a> <span class="s2">&quot;dragNodes&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="LineageHTML-213"><a href="#LineageHTML-213"><span class="linenos">213</span></a> <span class="s2">&quot;selectable&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="LineageHTML-214"><a href="#LineageHTML-214"><span class="linenos">214</span></a> <span class="p">},</span>
</span><span id="LineageHTML-215"><a href="#LineageHTML-215"><span class="linenos">215</span></a> <span class="s2">&quot;physics&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML-216"><a href="#LineageHTML-216"><span class="linenos">216</span></a> <span class="s2">&quot;enabled&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="LineageHTML-217"><a href="#LineageHTML-217"><span class="linenos">217</span></a> <span class="p">},</span>
</span><span id="LineageHTML-218"><a href="#LineageHTML-218"><span class="linenos">218</span></a> <span class="s2">&quot;edges&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML-219"><a href="#LineageHTML-219"><span class="linenos">219</span></a> <span class="s2">&quot;arrows&quot;</span><span class="p">:</span> <span class="s2">&quot;to&quot;</span><span class="p">,</span>
</span><span id="LineageHTML-220"><a href="#LineageHTML-220"><span class="linenos">220</span></a> <span class="p">},</span> </span><span id="LineageHTML-220"><a href="#LineageHTML-220"><span class="linenos">220</span></a> <span class="p">},</span>
</span><span id="LineageHTML-221"><a href="#LineageHTML-221"><span class="linenos">221</span></a> <span class="s2">&quot;nodes&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="LineageHTML-221"><a href="#LineageHTML-221"><span class="linenos">221</span></a> <span class="p">},</span>
</span><span id="LineageHTML-222"><a href="#LineageHTML-222"><span class="linenos">222</span></a> <span class="s2">&quot;font&quot;</span><span class="p">:</span> <span class="s2">&quot;20px monaco&quot;</span><span class="p">,</span> </span><span id="LineageHTML-222"><a href="#LineageHTML-222"><span class="linenos">222</span></a> <span class="s2">&quot;interaction&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML-223"><a href="#LineageHTML-223"><span class="linenos">223</span></a> <span class="s2">&quot;shape&quot;</span><span class="p">:</span> <span class="s2">&quot;box&quot;</span><span class="p">,</span> </span><span id="LineageHTML-223"><a href="#LineageHTML-223"><span class="linenos">223</span></a> <span class="s2">&quot;dragNodes&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="LineageHTML-224"><a href="#LineageHTML-224"><span class="linenos">224</span></a> <span class="s2">&quot;widthConstraint&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="LineageHTML-224"><a href="#LineageHTML-224"><span class="linenos">224</span></a> <span class="s2">&quot;selectable&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="LineageHTML-225"><a href="#LineageHTML-225"><span class="linenos">225</span></a> <span class="s2">&quot;maximum&quot;</span><span class="p">:</span> <span class="mi">300</span><span class="p">,</span> </span><span id="LineageHTML-225"><a href="#LineageHTML-225"><span class="linenos">225</span></a> <span class="p">},</span>
</span><span id="LineageHTML-226"><a href="#LineageHTML-226"><span class="linenos">226</span></a> <span class="p">},</span> </span><span id="LineageHTML-226"><a href="#LineageHTML-226"><span class="linenos">226</span></a> <span class="s2">&quot;physics&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML-227"><a href="#LineageHTML-227"><span class="linenos">227</span></a> <span class="p">},</span> </span><span id="LineageHTML-227"><a href="#LineageHTML-227"><span class="linenos">227</span></a> <span class="s2">&quot;enabled&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="LineageHTML-228"><a href="#LineageHTML-228"><span class="linenos">228</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">,</span> </span><span id="LineageHTML-228"><a href="#LineageHTML-228"><span class="linenos">228</span></a> <span class="p">},</span>
</span><span id="LineageHTML-229"><a href="#LineageHTML-229"><span class="linenos">229</span></a> <span class="p">}</span> </span><span id="LineageHTML-229"><a href="#LineageHTML-229"><span class="linenos">229</span></a> <span class="s2">&quot;edges&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML-230"><a href="#LineageHTML-230"><span class="linenos">230</span></a> </span><span id="LineageHTML-230"><a href="#LineageHTML-230"><span class="linenos">230</span></a> <span class="s2">&quot;arrows&quot;</span><span class="p">:</span> <span class="s2">&quot;to&quot;</span><span class="p">,</span>
</span><span id="LineageHTML-231"><a href="#LineageHTML-231"><span class="linenos">231</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span> </span><span id="LineageHTML-231"><a href="#LineageHTML-231"><span class="linenos">231</span></a> <span class="p">},</span>
</span><span id="LineageHTML-232"><a href="#LineageHTML-232"><span class="linenos">232</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span> <span class="o">=</span> <span class="p">[]</span> </span><span id="LineageHTML-232"><a href="#LineageHTML-232"><span class="linenos">232</span></a> <span class="s2">&quot;nodes&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML-233"><a href="#LineageHTML-233"><span class="linenos">233</span></a> </span><span id="LineageHTML-233"><a href="#LineageHTML-233"><span class="linenos">233</span></a> <span class="s2">&quot;font&quot;</span><span class="p">:</span> <span class="s2">&quot;20px monaco&quot;</span><span class="p">,</span>
</span><span id="LineageHTML-234"><a href="#LineageHTML-234"><span class="linenos">234</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span> </span><span id="LineageHTML-234"><a href="#LineageHTML-234"><span class="linenos">234</span></a> <span class="s2">&quot;shape&quot;</span><span class="p">:</span> <span class="s2">&quot;box&quot;</span><span class="p">,</span>
</span><span id="LineageHTML-235"><a href="#LineageHTML-235"><span class="linenos">235</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span> </span><span id="LineageHTML-235"><a href="#LineageHTML-235"><span class="linenos">235</span></a> <span class="s2">&quot;widthConstraint&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML-236"><a href="#LineageHTML-236"><span class="linenos">236</span></a> <span class="n">label</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">&quot;</span> </span><span id="LineageHTML-236"><a href="#LineageHTML-236"><span class="linenos">236</span></a> <span class="s2">&quot;maximum&quot;</span><span class="p">:</span> <span class="mi">300</span><span class="p">,</span>
</span><span id="LineageHTML-237"><a href="#LineageHTML-237"><span class="linenos">237</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;&lt;pre&gt;SELECT </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2"> FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">&lt;/pre&gt;&quot;</span> </span><span id="LineageHTML-237"><a href="#LineageHTML-237"><span class="linenos">237</span></a> <span class="p">},</span>
</span><span id="LineageHTML-238"><a href="#LineageHTML-238"><span class="linenos">238</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">1</span> </span><span id="LineageHTML-238"><a href="#LineageHTML-238"><span class="linenos">238</span></a> <span class="p">},</span>
</span><span id="LineageHTML-239"><a href="#LineageHTML-239"><span class="linenos">239</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="LineageHTML-239"><a href="#LineageHTML-239"><span class="linenos">239</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">,</span>
</span><span id="LineageHTML-240"><a href="#LineageHTML-240"><span class="linenos">240</span></a> <span class="n">label</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> </span><span id="LineageHTML-240"><a href="#LineageHTML-240"><span class="linenos">240</span></a> <span class="p">}</span>
</span><span id="LineageHTML-241"><a href="#LineageHTML-241"><span class="linenos">241</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">source</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span> </span><span id="LineageHTML-241"><a href="#LineageHTML-241"><span class="linenos">241</span></a>
</span><span id="LineageHTML-242"><a href="#LineageHTML-242"><span class="linenos">242</span></a> <span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Tag</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">n</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">&quot;&lt;b&gt;&quot;</span><span class="p">,</span> <span class="n">postfix</span><span class="o">=</span><span class="s2">&quot;&lt;/b&gt;&quot;</span><span class="p">)</span> </span><span id="LineageHTML-242"><a href="#LineageHTML-242"><span class="linenos">242</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="LineageHTML-243"><a href="#LineageHTML-243"><span class="linenos">243</span></a> <span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span> </span><span id="LineageHTML-243"><a href="#LineageHTML-243"><span class="linenos">243</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span> <span class="o">=</span> <span class="p">[]</span>
</span><span id="LineageHTML-244"><a href="#LineageHTML-244"><span class="linenos">244</span></a> <span class="k">else</span> <span class="n">n</span><span class="p">,</span> </span><span id="LineageHTML-244"><a href="#LineageHTML-244"><span class="linenos">244</span></a>
</span><span id="LineageHTML-245"><a href="#LineageHTML-245"><span class="linenos">245</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> </span><span id="LineageHTML-245"><a href="#LineageHTML-245"><span class="linenos">245</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
</span><span id="LineageHTML-246"><a href="#LineageHTML-246"><span class="linenos">246</span></a> <span class="p">)</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> </span><span id="LineageHTML-246"><a href="#LineageHTML-246"><span class="linenos">246</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="LineageHTML-247"><a href="#LineageHTML-247"><span class="linenos">247</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;&lt;pre&gt;</span><span class="si">{</span><span class="n">source</span><span class="si">}</span><span class="s2">&lt;/pre&gt;&quot;</span> </span><span id="LineageHTML-247"><a href="#LineageHTML-247"><span class="linenos">247</span></a> <span class="n">label</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">&quot;</span>
</span><span id="LineageHTML-248"><a href="#LineageHTML-248"><span class="linenos">248</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">0</span> </span><span id="LineageHTML-248"><a href="#LineageHTML-248"><span class="linenos">248</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;&lt;pre&gt;SELECT </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2"> FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">&lt;/pre&gt;&quot;</span>
</span><span id="LineageHTML-249"><a href="#LineageHTML-249"><span class="linenos">249</span></a> </span><span id="LineageHTML-249"><a href="#LineageHTML-249"><span class="linenos">249</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">1</span>
</span><span id="LineageHTML-250"><a href="#LineageHTML-250"><span class="linenos">250</span></a> <span class="n">node_id</span> <span class="o">=</span> <span class="nb">id</span><span class="p">(</span><span class="n">node</span><span class="p">)</span> </span><span id="LineageHTML-250"><a href="#LineageHTML-250"><span class="linenos">250</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="LineageHTML-251"><a href="#LineageHTML-251"><span class="linenos">251</span></a> </span><span id="LineageHTML-251"><a href="#LineageHTML-251"><span class="linenos">251</span></a> <span class="n">label</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="LineageHTML-252"><a href="#LineageHTML-252"><span class="linenos">252</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="p">[</span><span class="n">node_id</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span> </span><span id="LineageHTML-252"><a href="#LineageHTML-252"><span class="linenos">252</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">source</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
</span><span id="LineageHTML-253"><a href="#LineageHTML-253"><span class="linenos">253</span></a> <span class="s2">&quot;id&quot;</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> </span><span id="LineageHTML-253"><a href="#LineageHTML-253"><span class="linenos">253</span></a> <span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Tag</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">n</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">&quot;&lt;b&gt;&quot;</span><span class="p">,</span> <span class="n">postfix</span><span class="o">=</span><span class="s2">&quot;&lt;/b&gt;&quot;</span><span class="p">)</span>
</span><span id="LineageHTML-254"><a href="#LineageHTML-254"><span class="linenos">254</span></a> <span class="s2">&quot;label&quot;</span><span class="p">:</span> <span class="n">label</span><span class="p">,</span> </span><span id="LineageHTML-254"><a href="#LineageHTML-254"><span class="linenos">254</span></a> <span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span>
</span><span id="LineageHTML-255"><a href="#LineageHTML-255"><span class="linenos">255</span></a> <span class="s2">&quot;title&quot;</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span> </span><span id="LineageHTML-255"><a href="#LineageHTML-255"><span class="linenos">255</span></a> <span class="k">else</span> <span class="n">n</span><span class="p">,</span>
</span><span id="LineageHTML-256"><a href="#LineageHTML-256"><span class="linenos">256</span></a> <span class="s2">&quot;group&quot;</span><span class="p">:</span> <span class="n">group</span><span class="p">,</span> </span><span id="LineageHTML-256"><a href="#LineageHTML-256"><span class="linenos">256</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="LineageHTML-257"><a href="#LineageHTML-257"><span class="linenos">257</span></a> <span class="p">}</span> </span><span id="LineageHTML-257"><a href="#LineageHTML-257"><span class="linenos">257</span></a> <span class="p">)</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="LineageHTML-258"><a href="#LineageHTML-258"><span class="linenos">258</span></a> </span><span id="LineageHTML-258"><a href="#LineageHTML-258"><span class="linenos">258</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;&lt;pre&gt;</span><span class="si">{</span><span class="n">source</span><span class="si">}</span><span class="s2">&lt;/pre&gt;&quot;</span>
</span><span id="LineageHTML-259"><a href="#LineageHTML-259"><span class="linenos">259</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="p">:</span> </span><span id="LineageHTML-259"><a href="#LineageHTML-259"><span class="linenos">259</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">0</span>
</span><span id="LineageHTML-260"><a href="#LineageHTML-260"><span class="linenos">260</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s2">&quot;from&quot;</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> <span class="s2">&quot;to&quot;</span><span class="p">:</span> <span class="nb">id</span><span class="p">(</span><span class="n">d</span><span class="p">)})</span> </span><span id="LineageHTML-260"><a href="#LineageHTML-260"><span class="linenos">260</span></a>
</span><span id="LineageHTML-261"><a href="#LineageHTML-261"><span class="linenos">261</span></a> </span><span id="LineageHTML-261"><a href="#LineageHTML-261"><span class="linenos">261</span></a> <span class="n">node_id</span> <span class="o">=</span> <span class="nb">id</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="LineageHTML-262"><a href="#LineageHTML-262"><span class="linenos">262</span></a> <span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> </span><span id="LineageHTML-262"><a href="#LineageHTML-262"><span class="linenos">262</span></a>
</span><span id="LineageHTML-263"><a href="#LineageHTML-263"><span class="linenos">263</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="o">.</span><span class="n">values</span><span class="p">()))</span> </span><span id="LineageHTML-263"><a href="#LineageHTML-263"><span class="linenos">263</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="p">[</span><span class="n">node_id</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="LineageHTML-264"><a href="#LineageHTML-264"><span class="linenos">264</span></a> <span class="n">edges</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="p">)</span> </span><span id="LineageHTML-264"><a href="#LineageHTML-264"><span class="linenos">264</span></a> <span class="s2">&quot;id&quot;</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span>
</span><span id="LineageHTML-265"><a href="#LineageHTML-265"><span class="linenos">265</span></a> <span class="n">options</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">options</span><span class="p">)</span> </span><span id="LineageHTML-265"><a href="#LineageHTML-265"><span class="linenos">265</span></a> <span class="s2">&quot;label&quot;</span><span class="p">:</span> <span class="n">label</span><span class="p">,</span>
</span><span id="LineageHTML-266"><a href="#LineageHTML-266"><span class="linenos">266</span></a> <span class="n">imports</span> <span class="o">=</span> <span class="p">(</span> </span><span id="LineageHTML-266"><a href="#LineageHTML-266"><span class="linenos">266</span></a> <span class="s2">&quot;title&quot;</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span>
</span><span id="LineageHTML-267"><a href="#LineageHTML-267"><span class="linenos">267</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;&lt;script type=&quot;text/javascript&quot; src=&quot;https://unpkg.com/vis-data@latest/peer/umd/vis-data.min.js&quot;&gt;&lt;/script&gt;</span> </span><span id="LineageHTML-267"><a href="#LineageHTML-267"><span class="linenos">267</span></a> <span class="s2">&quot;group&quot;</span><span class="p">:</span> <span class="n">group</span><span class="p">,</span>
</span><span id="LineageHTML-268"><a href="#LineageHTML-268"><span class="linenos">268</span></a><span class="sd"> &lt;script type=&quot;text/javascript&quot; src=&quot;https://unpkg.com/vis-network@latest/peer/umd/vis-network.min.js&quot;&gt;&lt;/script&gt;</span> </span><span id="LineageHTML-268"><a href="#LineageHTML-268"><span class="linenos">268</span></a> <span class="p">}</span>
</span><span id="LineageHTML-269"><a href="#LineageHTML-269"><span class="linenos">269</span></a><span class="sd"> &lt;link rel=&quot;stylesheet&quot; type=&quot;text/css&quot; href=&quot;https://unpkg.com/vis-network/styles/vis-network.min.css&quot; /&gt;&quot;&quot;&quot;</span> </span><span id="LineageHTML-269"><a href="#LineageHTML-269"><span class="linenos">269</span></a>
</span><span id="LineageHTML-270"><a href="#LineageHTML-270"><span class="linenos">270</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span> </span><span id="LineageHTML-270"><a href="#LineageHTML-270"><span class="linenos">270</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="p">:</span>
</span><span id="LineageHTML-271"><a href="#LineageHTML-271"><span class="linenos">271</span></a> <span class="k">else</span> <span class="s2">&quot;&quot;</span> </span><span id="LineageHTML-271"><a href="#LineageHTML-271"><span class="linenos">271</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s2">&quot;from&quot;</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> <span class="s2">&quot;to&quot;</span><span class="p">:</span> <span class="nb">id</span><span class="p">(</span><span class="n">d</span><span class="p">)})</span>
</span><span id="LineageHTML-272"><a href="#LineageHTML-272"><span class="linenos">272</span></a> <span class="p">)</span> </span><span id="LineageHTML-272"><a href="#LineageHTML-272"><span class="linenos">272</span></a>
</span><span id="LineageHTML-273"><a href="#LineageHTML-273"><span class="linenos">273</span></a> </span><span id="LineageHTML-273"><a href="#LineageHTML-273"><span class="linenos">273</span></a> <span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
</span><span id="LineageHTML-274"><a href="#LineageHTML-274"><span class="linenos">274</span></a> <span class="k">return</span> <span class="sa">f</span><span class="s2">&quot;&quot;&quot;&lt;div&gt;</span> </span><span id="LineageHTML-274"><a href="#LineageHTML-274"><span class="linenos">274</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="o">.</span><span class="n">values</span><span class="p">()))</span>
</span><span id="LineageHTML-275"><a href="#LineageHTML-275"><span class="linenos">275</span></a><span class="s2"> &lt;div id=&quot;sqlglot-lineage&quot;&gt;&lt;/div&gt;</span> </span><span id="LineageHTML-275"><a href="#LineageHTML-275"><span class="linenos">275</span></a> <span class="n">edges</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="p">)</span>
</span><span id="LineageHTML-276"><a href="#LineageHTML-276"><span class="linenos">276</span></a><span class="s2"> </span><span class="si">{</span><span class="n">imports</span><span class="si">}</span> </span><span id="LineageHTML-276"><a href="#LineageHTML-276"><span class="linenos">276</span></a> <span class="n">options</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">options</span><span class="p">)</span>
</span><span id="LineageHTML-277"><a href="#LineageHTML-277"><span class="linenos">277</span></a><span class="s2"> &lt;script type=&quot;text/javascript&quot;&gt;</span> </span><span id="LineageHTML-277"><a href="#LineageHTML-277"><span class="linenos">277</span></a> <span class="n">imports</span> <span class="o">=</span> <span class="p">(</span>
</span><span id="LineageHTML-278"><a href="#LineageHTML-278"><span class="linenos">278</span></a><span class="s2"> var nodes = new vis.DataSet(</span><span class="si">{</span><span class="n">nodes</span><span class="si">}</span><span class="s2">)</span> </span><span id="LineageHTML-278"><a href="#LineageHTML-278"><span class="linenos">278</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;&lt;script type=&quot;text/javascript&quot; src=&quot;https://unpkg.com/vis-data@latest/peer/umd/vis-data.min.js&quot;&gt;&lt;/script&gt;</span>
</span><span id="LineageHTML-279"><a href="#LineageHTML-279"><span class="linenos">279</span></a><span class="s2"> nodes.forEach(row =&gt; row[&quot;title&quot;] = new DOMParser().parseFromString(row[&quot;title&quot;], &quot;text/html&quot;).body.childNodes[0])</span> </span><span id="LineageHTML-279"><a href="#LineageHTML-279"><span class="linenos">279</span></a><span class="sd"> &lt;script type=&quot;text/javascript&quot; src=&quot;https://unpkg.com/vis-network@latest/peer/umd/vis-network.min.js&quot;&gt;&lt;/script&gt;</span>
</span><span id="LineageHTML-280"><a href="#LineageHTML-280"><span class="linenos">280</span></a> </span><span id="LineageHTML-280"><a href="#LineageHTML-280"><span class="linenos">280</span></a><span class="sd"> &lt;link rel=&quot;stylesheet&quot; type=&quot;text/css&quot; href=&quot;https://unpkg.com/vis-network/styles/vis-network.min.css&quot; /&gt;&quot;&quot;&quot;</span>
</span><span id="LineageHTML-281"><a href="#LineageHTML-281"><span class="linenos">281</span></a><span class="s2"> new vis.Network(</span> </span><span id="LineageHTML-281"><a href="#LineageHTML-281"><span class="linenos">281</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span>
</span><span id="LineageHTML-282"><a href="#LineageHTML-282"><span class="linenos">282</span></a><span class="s2"> document.getElementById(&quot;sqlglot-lineage&quot;),</span> </span><span id="LineageHTML-282"><a href="#LineageHTML-282"><span class="linenos">282</span></a> <span class="k">else</span> <span class="s2">&quot;&quot;</span>
</span><span id="LineageHTML-283"><a href="#LineageHTML-283"><span class="linenos">283</span></a><span class="s2"> </span><span class="se">{{</span> </span><span id="LineageHTML-283"><a href="#LineageHTML-283"><span class="linenos">283</span></a> <span class="p">)</span>
</span><span id="LineageHTML-284"><a href="#LineageHTML-284"><span class="linenos">284</span></a><span class="s2"> nodes: nodes,</span> </span><span id="LineageHTML-284"><a href="#LineageHTML-284"><span class="linenos">284</span></a>
</span><span id="LineageHTML-285"><a href="#LineageHTML-285"><span class="linenos">285</span></a><span class="s2"> edges: new vis.DataSet(</span><span class="si">{</span><span class="n">edges</span><span class="si">}</span><span class="s2">)</span> </span><span id="LineageHTML-285"><a href="#LineageHTML-285"><span class="linenos">285</span></a> <span class="k">return</span> <span class="sa">f</span><span class="s2">&quot;&quot;&quot;&lt;div&gt;</span>
</span><span id="LineageHTML-286"><a href="#LineageHTML-286"><span class="linenos">286</span></a><span class="s2"> </span><span class="se">}}</span><span class="s2">,</span> </span><span id="LineageHTML-286"><a href="#LineageHTML-286"><span class="linenos">286</span></a><span class="s2"> &lt;div id=&quot;sqlglot-lineage&quot;&gt;&lt;/div&gt;</span>
</span><span id="LineageHTML-287"><a href="#LineageHTML-287"><span class="linenos">287</span></a><span class="s2"> </span><span class="si">{</span><span class="n">options</span><span class="si">}</span><span class="s2">,</span> </span><span id="LineageHTML-287"><a href="#LineageHTML-287"><span class="linenos">287</span></a><span class="s2"> </span><span class="si">{</span><span class="n">imports</span><span class="si">}</span>
</span><span id="LineageHTML-288"><a href="#LineageHTML-288"><span class="linenos">288</span></a><span class="s2"> )</span> </span><span id="LineageHTML-288"><a href="#LineageHTML-288"><span class="linenos">288</span></a><span class="s2"> &lt;script type=&quot;text/javascript&quot;&gt;</span>
</span><span id="LineageHTML-289"><a href="#LineageHTML-289"><span class="linenos">289</span></a><span class="s2"> &lt;/script&gt;</span> </span><span id="LineageHTML-289"><a href="#LineageHTML-289"><span class="linenos">289</span></a><span class="s2"> var nodes = new vis.DataSet(</span><span class="si">{</span><span class="n">nodes</span><span class="si">}</span><span class="s2">)</span>
</span><span id="LineageHTML-290"><a href="#LineageHTML-290"><span class="linenos">290</span></a><span class="s2">&lt;/div&gt;&quot;&quot;&quot;</span> </span><span id="LineageHTML-290"><a href="#LineageHTML-290"><span class="linenos">290</span></a><span class="s2"> nodes.forEach(row =&gt; row[&quot;title&quot;] = new DOMParser().parseFromString(row[&quot;title&quot;], &quot;text/html&quot;).body.childNodes[0])</span>
</span><span id="LineageHTML-291"><a href="#LineageHTML-291"><span class="linenos">291</span></a> </span><span id="LineageHTML-291"><a href="#LineageHTML-291"><span class="linenos">291</span></a>
</span><span id="LineageHTML-292"><a href="#LineageHTML-292"><span class="linenos">292</span></a> <span class="k">def</span> <span class="nf">_repr_html_</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span> </span><span id="LineageHTML-292"><a href="#LineageHTML-292"><span class="linenos">292</span></a><span class="s2"> new vis.Network(</span>
</span><span id="LineageHTML-293"><a href="#LineageHTML-293"><span class="linenos">293</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="fm">__str__</span><span class="p">()</span> </span><span id="LineageHTML-293"><a href="#LineageHTML-293"><span class="linenos">293</span></a><span class="s2"> document.getElementById(&quot;sqlglot-lineage&quot;),</span>
</span><span id="LineageHTML-294"><a href="#LineageHTML-294"><span class="linenos">294</span></a><span class="s2"> </span><span class="se">{{</span>
</span><span id="LineageHTML-295"><a href="#LineageHTML-295"><span class="linenos">295</span></a><span class="s2"> nodes: nodes,</span>
</span><span id="LineageHTML-296"><a href="#LineageHTML-296"><span class="linenos">296</span></a><span class="s2"> edges: new vis.DataSet(</span><span class="si">{</span><span class="n">edges</span><span class="si">}</span><span class="s2">)</span>
</span><span id="LineageHTML-297"><a href="#LineageHTML-297"><span class="linenos">297</span></a><span class="s2"> </span><span class="se">}}</span><span class="s2">,</span>
</span><span id="LineageHTML-298"><a href="#LineageHTML-298"><span class="linenos">298</span></a><span class="s2"> </span><span class="si">{</span><span class="n">options</span><span class="si">}</span><span class="s2">,</span>
</span><span id="LineageHTML-299"><a href="#LineageHTML-299"><span class="linenos">299</span></a><span class="s2"> )</span>
</span><span id="LineageHTML-300"><a href="#LineageHTML-300"><span class="linenos">300</span></a><span class="s2"> &lt;/script&gt;</span>
</span><span id="LineageHTML-301"><a href="#LineageHTML-301"><span class="linenos">301</span></a><span class="s2">&lt;/div&gt;&quot;&quot;&quot;</span>
</span><span id="LineageHTML-302"><a href="#LineageHTML-302"><span class="linenos">302</span></a>
</span><span id="LineageHTML-303"><a href="#LineageHTML-303"><span class="linenos">303</span></a> <span class="k">def</span> <span class="nf">_repr_html_</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
</span><span id="LineageHTML-304"><a href="#LineageHTML-304"><span class="linenos">304</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="fm">__str__</span><span class="p">()</span>
</span></pre></div> </span></pre></div>
@ -876,76 +898,76 @@
</div> </div>
<a class="headerlink" href="#LineageHTML.__init__"></a> <a class="headerlink" href="#LineageHTML.__init__"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="LineageHTML.__init__-191"><a href="#LineageHTML.__init__-191"><span class="linenos">191</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span> <div class="pdoc-code codehilite"><pre><span></span><span id="LineageHTML.__init__-202"><a href="#LineageHTML.__init__-202"><span class="linenos">202</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span>
</span><span id="LineageHTML.__init__-192"><a href="#LineageHTML.__init__-192"><span class="linenos">192</span></a> <span class="bp">self</span><span class="p">,</span> </span><span id="LineageHTML.__init__-203"><a href="#LineageHTML.__init__-203"><span class="linenos">203</span></a> <span class="bp">self</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-193"><a href="#LineageHTML.__init__-193"><span class="linenos">193</span></a> <span class="n">node</span><span class="p">:</span> <span class="n">Node</span><span class="p">,</span> </span><span id="LineageHTML.__init__-204"><a href="#LineageHTML.__init__-204"><span class="linenos">204</span></a> <span class="n">node</span><span class="p">:</span> <span class="n">Node</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-194"><a href="#LineageHTML.__init__-194"><span class="linenos">194</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> </span><span id="LineageHTML.__init__-205"><a href="#LineageHTML.__init__-205"><span class="linenos">205</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-195"><a href="#LineageHTML.__init__-195"><span class="linenos">195</span></a> <span class="n">imports</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span> </span><span id="LineageHTML.__init__-206"><a href="#LineageHTML.__init__-206"><span class="linenos">206</span></a> <span class="n">imports</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-196"><a href="#LineageHTML.__init__-196"><span class="linenos">196</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span> </span><span id="LineageHTML.__init__-207"><a href="#LineageHTML.__init__-207"><span class="linenos">207</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-197"><a href="#LineageHTML.__init__-197"><span class="linenos">197</span></a> <span class="p">):</span> </span><span id="LineageHTML.__init__-208"><a href="#LineageHTML.__init__-208"><span class="linenos">208</span></a> <span class="p">):</span>
</span><span id="LineageHTML.__init__-198"><a href="#LineageHTML.__init__-198"><span class="linenos">198</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">node</span> <span class="o">=</span> <span class="n">node</span> </span><span id="LineageHTML.__init__-209"><a href="#LineageHTML.__init__-209"><span class="linenos">209</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">node</span> <span class="o">=</span> <span class="n">node</span>
</span><span id="LineageHTML.__init__-199"><a href="#LineageHTML.__init__-199"><span class="linenos">199</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span> <span class="o">=</span> <span class="n">imports</span> </span><span id="LineageHTML.__init__-210"><a href="#LineageHTML.__init__-210"><span class="linenos">210</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span> <span class="o">=</span> <span class="n">imports</span>
</span><span id="LineageHTML.__init__-200"><a href="#LineageHTML.__init__-200"><span class="linenos">200</span></a> </span><span id="LineageHTML.__init__-211"><a href="#LineageHTML.__init__-211"><span class="linenos">211</span></a>
</span><span id="LineageHTML.__init__-201"><a href="#LineageHTML.__init__-201"><span class="linenos">201</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">options</span> <span class="o">=</span> <span class="p">{</span> </span><span id="LineageHTML.__init__-212"><a href="#LineageHTML.__init__-212"><span class="linenos">212</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="LineageHTML.__init__-202"><a href="#LineageHTML.__init__-202"><span class="linenos">202</span></a> <span class="s2">&quot;height&quot;</span><span class="p">:</span> <span class="s2">&quot;500px&quot;</span><span class="p">,</span> </span><span id="LineageHTML.__init__-213"><a href="#LineageHTML.__init__-213"><span class="linenos">213</span></a> <span class="s2">&quot;height&quot;</span><span class="p">:</span> <span class="s2">&quot;500px&quot;</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-203"><a href="#LineageHTML.__init__-203"><span class="linenos">203</span></a> <span class="s2">&quot;width&quot;</span><span class="p">:</span> <span class="s2">&quot;100%&quot;</span><span class="p">,</span> </span><span id="LineageHTML.__init__-214"><a href="#LineageHTML.__init__-214"><span class="linenos">214</span></a> <span class="s2">&quot;width&quot;</span><span class="p">:</span> <span class="s2">&quot;100%&quot;</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-204"><a href="#LineageHTML.__init__-204"><span class="linenos">204</span></a> <span class="s2">&quot;layout&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="LineageHTML.__init__-215"><a href="#LineageHTML.__init__-215"><span class="linenos">215</span></a> <span class="s2">&quot;layout&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML.__init__-205"><a href="#LineageHTML.__init__-205"><span class="linenos">205</span></a> <span class="s2">&quot;hierarchical&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="LineageHTML.__init__-216"><a href="#LineageHTML.__init__-216"><span class="linenos">216</span></a> <span class="s2">&quot;hierarchical&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML.__init__-206"><a href="#LineageHTML.__init__-206"><span class="linenos">206</span></a> <span class="s2">&quot;enabled&quot;</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span> </span><span id="LineageHTML.__init__-217"><a href="#LineageHTML.__init__-217"><span class="linenos">217</span></a> <span class="s2">&quot;enabled&quot;</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-207"><a href="#LineageHTML.__init__-207"><span class="linenos">207</span></a> <span class="s2">&quot;nodeSpacing&quot;</span><span class="p">:</span> <span class="mi">200</span><span class="p">,</span> </span><span id="LineageHTML.__init__-218"><a href="#LineageHTML.__init__-218"><span class="linenos">218</span></a> <span class="s2">&quot;nodeSpacing&quot;</span><span class="p">:</span> <span class="mi">200</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-208"><a href="#LineageHTML.__init__-208"><span class="linenos">208</span></a> <span class="s2">&quot;sortMethod&quot;</span><span class="p">:</span> <span class="s2">&quot;directed&quot;</span><span class="p">,</span> </span><span id="LineageHTML.__init__-219"><a href="#LineageHTML.__init__-219"><span class="linenos">219</span></a> <span class="s2">&quot;sortMethod&quot;</span><span class="p">:</span> <span class="s2">&quot;directed&quot;</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-209"><a href="#LineageHTML.__init__-209"><span class="linenos">209</span></a> <span class="p">},</span>
</span><span id="LineageHTML.__init__-210"><a href="#LineageHTML.__init__-210"><span class="linenos">210</span></a> <span class="p">},</span>
</span><span id="LineageHTML.__init__-211"><a href="#LineageHTML.__init__-211"><span class="linenos">211</span></a> <span class="s2">&quot;interaction&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML.__init__-212"><a href="#LineageHTML.__init__-212"><span class="linenos">212</span></a> <span class="s2">&quot;dragNodes&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-213"><a href="#LineageHTML.__init__-213"><span class="linenos">213</span></a> <span class="s2">&quot;selectable&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-214"><a href="#LineageHTML.__init__-214"><span class="linenos">214</span></a> <span class="p">},</span>
</span><span id="LineageHTML.__init__-215"><a href="#LineageHTML.__init__-215"><span class="linenos">215</span></a> <span class="s2">&quot;physics&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML.__init__-216"><a href="#LineageHTML.__init__-216"><span class="linenos">216</span></a> <span class="s2">&quot;enabled&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-217"><a href="#LineageHTML.__init__-217"><span class="linenos">217</span></a> <span class="p">},</span>
</span><span id="LineageHTML.__init__-218"><a href="#LineageHTML.__init__-218"><span class="linenos">218</span></a> <span class="s2">&quot;edges&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML.__init__-219"><a href="#LineageHTML.__init__-219"><span class="linenos">219</span></a> <span class="s2">&quot;arrows&quot;</span><span class="p">:</span> <span class="s2">&quot;to&quot;</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-220"><a href="#LineageHTML.__init__-220"><span class="linenos">220</span></a> <span class="p">},</span> </span><span id="LineageHTML.__init__-220"><a href="#LineageHTML.__init__-220"><span class="linenos">220</span></a> <span class="p">},</span>
</span><span id="LineageHTML.__init__-221"><a href="#LineageHTML.__init__-221"><span class="linenos">221</span></a> <span class="s2">&quot;nodes&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="LineageHTML.__init__-221"><a href="#LineageHTML.__init__-221"><span class="linenos">221</span></a> <span class="p">},</span>
</span><span id="LineageHTML.__init__-222"><a href="#LineageHTML.__init__-222"><span class="linenos">222</span></a> <span class="s2">&quot;font&quot;</span><span class="p">:</span> <span class="s2">&quot;20px monaco&quot;</span><span class="p">,</span> </span><span id="LineageHTML.__init__-222"><a href="#LineageHTML.__init__-222"><span class="linenos">222</span></a> <span class="s2">&quot;interaction&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML.__init__-223"><a href="#LineageHTML.__init__-223"><span class="linenos">223</span></a> <span class="s2">&quot;shape&quot;</span><span class="p">:</span> <span class="s2">&quot;box&quot;</span><span class="p">,</span> </span><span id="LineageHTML.__init__-223"><a href="#LineageHTML.__init__-223"><span class="linenos">223</span></a> <span class="s2">&quot;dragNodes&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-224"><a href="#LineageHTML.__init__-224"><span class="linenos">224</span></a> <span class="s2">&quot;widthConstraint&quot;</span><span class="p">:</span> <span class="p">{</span> </span><span id="LineageHTML.__init__-224"><a href="#LineageHTML.__init__-224"><span class="linenos">224</span></a> <span class="s2">&quot;selectable&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-225"><a href="#LineageHTML.__init__-225"><span class="linenos">225</span></a> <span class="s2">&quot;maximum&quot;</span><span class="p">:</span> <span class="mi">300</span><span class="p">,</span> </span><span id="LineageHTML.__init__-225"><a href="#LineageHTML.__init__-225"><span class="linenos">225</span></a> <span class="p">},</span>
</span><span id="LineageHTML.__init__-226"><a href="#LineageHTML.__init__-226"><span class="linenos">226</span></a> <span class="p">},</span> </span><span id="LineageHTML.__init__-226"><a href="#LineageHTML.__init__-226"><span class="linenos">226</span></a> <span class="s2">&quot;physics&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML.__init__-227"><a href="#LineageHTML.__init__-227"><span class="linenos">227</span></a> <span class="p">},</span> </span><span id="LineageHTML.__init__-227"><a href="#LineageHTML.__init__-227"><span class="linenos">227</span></a> <span class="s2">&quot;enabled&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-228"><a href="#LineageHTML.__init__-228"><span class="linenos">228</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">,</span> </span><span id="LineageHTML.__init__-228"><a href="#LineageHTML.__init__-228"><span class="linenos">228</span></a> <span class="p">},</span>
</span><span id="LineageHTML.__init__-229"><a href="#LineageHTML.__init__-229"><span class="linenos">229</span></a> <span class="p">}</span> </span><span id="LineageHTML.__init__-229"><a href="#LineageHTML.__init__-229"><span class="linenos">229</span></a> <span class="s2">&quot;edges&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML.__init__-230"><a href="#LineageHTML.__init__-230"><span class="linenos">230</span></a> </span><span id="LineageHTML.__init__-230"><a href="#LineageHTML.__init__-230"><span class="linenos">230</span></a> <span class="s2">&quot;arrows&quot;</span><span class="p">:</span> <span class="s2">&quot;to&quot;</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-231"><a href="#LineageHTML.__init__-231"><span class="linenos">231</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span> </span><span id="LineageHTML.__init__-231"><a href="#LineageHTML.__init__-231"><span class="linenos">231</span></a> <span class="p">},</span>
</span><span id="LineageHTML.__init__-232"><a href="#LineageHTML.__init__-232"><span class="linenos">232</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span> <span class="o">=</span> <span class="p">[]</span> </span><span id="LineageHTML.__init__-232"><a href="#LineageHTML.__init__-232"><span class="linenos">232</span></a> <span class="s2">&quot;nodes&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML.__init__-233"><a href="#LineageHTML.__init__-233"><span class="linenos">233</span></a> </span><span id="LineageHTML.__init__-233"><a href="#LineageHTML.__init__-233"><span class="linenos">233</span></a> <span class="s2">&quot;font&quot;</span><span class="p">:</span> <span class="s2">&quot;20px monaco&quot;</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-234"><a href="#LineageHTML.__init__-234"><span class="linenos">234</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span> </span><span id="LineageHTML.__init__-234"><a href="#LineageHTML.__init__-234"><span class="linenos">234</span></a> <span class="s2">&quot;shape&quot;</span><span class="p">:</span> <span class="s2">&quot;box&quot;</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-235"><a href="#LineageHTML.__init__-235"><span class="linenos">235</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span> </span><span id="LineageHTML.__init__-235"><a href="#LineageHTML.__init__-235"><span class="linenos">235</span></a> <span class="s2">&quot;widthConstraint&quot;</span><span class="p">:</span> <span class="p">{</span>
</span><span id="LineageHTML.__init__-236"><a href="#LineageHTML.__init__-236"><span class="linenos">236</span></a> <span class="n">label</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">&quot;</span> </span><span id="LineageHTML.__init__-236"><a href="#LineageHTML.__init__-236"><span class="linenos">236</span></a> <span class="s2">&quot;maximum&quot;</span><span class="p">:</span> <span class="mi">300</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-237"><a href="#LineageHTML.__init__-237"><span class="linenos">237</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;&lt;pre&gt;SELECT </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2"> FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">&lt;/pre&gt;&quot;</span> </span><span id="LineageHTML.__init__-237"><a href="#LineageHTML.__init__-237"><span class="linenos">237</span></a> <span class="p">},</span>
</span><span id="LineageHTML.__init__-238"><a href="#LineageHTML.__init__-238"><span class="linenos">238</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">1</span> </span><span id="LineageHTML.__init__-238"><a href="#LineageHTML.__init__-238"><span class="linenos">238</span></a> <span class="p">},</span>
</span><span id="LineageHTML.__init__-239"><a href="#LineageHTML.__init__-239"><span class="linenos">239</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="LineageHTML.__init__-239"><a href="#LineageHTML.__init__-239"><span class="linenos">239</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-240"><a href="#LineageHTML.__init__-240"><span class="linenos">240</span></a> <span class="n">label</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> </span><span id="LineageHTML.__init__-240"><a href="#LineageHTML.__init__-240"><span class="linenos">240</span></a> <span class="p">}</span>
</span><span id="LineageHTML.__init__-241"><a href="#LineageHTML.__init__-241"><span class="linenos">241</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">source</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span> </span><span id="LineageHTML.__init__-241"><a href="#LineageHTML.__init__-241"><span class="linenos">241</span></a>
</span><span id="LineageHTML.__init__-242"><a href="#LineageHTML.__init__-242"><span class="linenos">242</span></a> <span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Tag</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">n</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">&quot;&lt;b&gt;&quot;</span><span class="p">,</span> <span class="n">postfix</span><span class="o">=</span><span class="s2">&quot;&lt;/b&gt;&quot;</span><span class="p">)</span> </span><span id="LineageHTML.__init__-242"><a href="#LineageHTML.__init__-242"><span class="linenos">242</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
</span><span id="LineageHTML.__init__-243"><a href="#LineageHTML.__init__-243"><span class="linenos">243</span></a> <span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span> </span><span id="LineageHTML.__init__-243"><a href="#LineageHTML.__init__-243"><span class="linenos">243</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span> <span class="o">=</span> <span class="p">[]</span>
</span><span id="LineageHTML.__init__-244"><a href="#LineageHTML.__init__-244"><span class="linenos">244</span></a> <span class="k">else</span> <span class="n">n</span><span class="p">,</span> </span><span id="LineageHTML.__init__-244"><a href="#LineageHTML.__init__-244"><span class="linenos">244</span></a>
</span><span id="LineageHTML.__init__-245"><a href="#LineageHTML.__init__-245"><span class="linenos">245</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> </span><span id="LineageHTML.__init__-245"><a href="#LineageHTML.__init__-245"><span class="linenos">245</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
</span><span id="LineageHTML.__init__-246"><a href="#LineageHTML.__init__-246"><span class="linenos">246</span></a> <span class="p">)</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span> </span><span id="LineageHTML.__init__-246"><a href="#LineageHTML.__init__-246"><span class="linenos">246</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
</span><span id="LineageHTML.__init__-247"><a href="#LineageHTML.__init__-247"><span class="linenos">247</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;&lt;pre&gt;</span><span class="si">{</span><span class="n">source</span><span class="si">}</span><span class="s2">&lt;/pre&gt;&quot;</span> </span><span id="LineageHTML.__init__-247"><a href="#LineageHTML.__init__-247"><span class="linenos">247</span></a> <span class="n">label</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">&quot;</span>
</span><span id="LineageHTML.__init__-248"><a href="#LineageHTML.__init__-248"><span class="linenos">248</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">0</span> </span><span id="LineageHTML.__init__-248"><a href="#LineageHTML.__init__-248"><span class="linenos">248</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;&lt;pre&gt;SELECT </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2"> FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">&lt;/pre&gt;&quot;</span>
</span><span id="LineageHTML.__init__-249"><a href="#LineageHTML.__init__-249"><span class="linenos">249</span></a> </span><span id="LineageHTML.__init__-249"><a href="#LineageHTML.__init__-249"><span class="linenos">249</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">1</span>
</span><span id="LineageHTML.__init__-250"><a href="#LineageHTML.__init__-250"><span class="linenos">250</span></a> <span class="n">node_id</span> <span class="o">=</span> <span class="nb">id</span><span class="p">(</span><span class="n">node</span><span class="p">)</span> </span><span id="LineageHTML.__init__-250"><a href="#LineageHTML.__init__-250"><span class="linenos">250</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="LineageHTML.__init__-251"><a href="#LineageHTML.__init__-251"><span class="linenos">251</span></a> </span><span id="LineageHTML.__init__-251"><a href="#LineageHTML.__init__-251"><span class="linenos">251</span></a> <span class="n">label</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="LineageHTML.__init__-252"><a href="#LineageHTML.__init__-252"><span class="linenos">252</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="p">[</span><span class="n">node_id</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span> </span><span id="LineageHTML.__init__-252"><a href="#LineageHTML.__init__-252"><span class="linenos">252</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">source</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
</span><span id="LineageHTML.__init__-253"><a href="#LineageHTML.__init__-253"><span class="linenos">253</span></a> <span class="s2">&quot;id&quot;</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> </span><span id="LineageHTML.__init__-253"><a href="#LineageHTML.__init__-253"><span class="linenos">253</span></a> <span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Tag</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">n</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">&quot;&lt;b&gt;&quot;</span><span class="p">,</span> <span class="n">postfix</span><span class="o">=</span><span class="s2">&quot;&lt;/b&gt;&quot;</span><span class="p">)</span>
</span><span id="LineageHTML.__init__-254"><a href="#LineageHTML.__init__-254"><span class="linenos">254</span></a> <span class="s2">&quot;label&quot;</span><span class="p">:</span> <span class="n">label</span><span class="p">,</span> </span><span id="LineageHTML.__init__-254"><a href="#LineageHTML.__init__-254"><span class="linenos">254</span></a> <span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span>
</span><span id="LineageHTML.__init__-255"><a href="#LineageHTML.__init__-255"><span class="linenos">255</span></a> <span class="s2">&quot;title&quot;</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span> </span><span id="LineageHTML.__init__-255"><a href="#LineageHTML.__init__-255"><span class="linenos">255</span></a> <span class="k">else</span> <span class="n">n</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-256"><a href="#LineageHTML.__init__-256"><span class="linenos">256</span></a> <span class="s2">&quot;group&quot;</span><span class="p">:</span> <span class="n">group</span><span class="p">,</span> </span><span id="LineageHTML.__init__-256"><a href="#LineageHTML.__init__-256"><span class="linenos">256</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-257"><a href="#LineageHTML.__init__-257"><span class="linenos">257</span></a> <span class="p">}</span> </span><span id="LineageHTML.__init__-257"><a href="#LineageHTML.__init__-257"><span class="linenos">257</span></a> <span class="p">)</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="LineageHTML.__init__-258"><a href="#LineageHTML.__init__-258"><span class="linenos">258</span></a> </span><span id="LineageHTML.__init__-258"><a href="#LineageHTML.__init__-258"><span class="linenos">258</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;&lt;pre&gt;</span><span class="si">{</span><span class="n">source</span><span class="si">}</span><span class="s2">&lt;/pre&gt;&quot;</span>
</span><span id="LineageHTML.__init__-259"><a href="#LineageHTML.__init__-259"><span class="linenos">259</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="p">:</span> </span><span id="LineageHTML.__init__-259"><a href="#LineageHTML.__init__-259"><span class="linenos">259</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">0</span>
</span><span id="LineageHTML.__init__-260"><a href="#LineageHTML.__init__-260"><span class="linenos">260</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s2">&quot;from&quot;</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> <span class="s2">&quot;to&quot;</span><span class="p">:</span> <span class="nb">id</span><span class="p">(</span><span class="n">d</span><span class="p">)})</span> </span><span id="LineageHTML.__init__-260"><a href="#LineageHTML.__init__-260"><span class="linenos">260</span></a>
</span><span id="LineageHTML.__init__-261"><a href="#LineageHTML.__init__-261"><span class="linenos">261</span></a> <span class="n">node_id</span> <span class="o">=</span> <span class="nb">id</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="LineageHTML.__init__-262"><a href="#LineageHTML.__init__-262"><span class="linenos">262</span></a>
</span><span id="LineageHTML.__init__-263"><a href="#LineageHTML.__init__-263"><span class="linenos">263</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="p">[</span><span class="n">node_id</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
</span><span id="LineageHTML.__init__-264"><a href="#LineageHTML.__init__-264"><span class="linenos">264</span></a> <span class="s2">&quot;id&quot;</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-265"><a href="#LineageHTML.__init__-265"><span class="linenos">265</span></a> <span class="s2">&quot;label&quot;</span><span class="p">:</span> <span class="n">label</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-266"><a href="#LineageHTML.__init__-266"><span class="linenos">266</span></a> <span class="s2">&quot;title&quot;</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-267"><a href="#LineageHTML.__init__-267"><span class="linenos">267</span></a> <span class="s2">&quot;group&quot;</span><span class="p">:</span> <span class="n">group</span><span class="p">,</span>
</span><span id="LineageHTML.__init__-268"><a href="#LineageHTML.__init__-268"><span class="linenos">268</span></a> <span class="p">}</span>
</span><span id="LineageHTML.__init__-269"><a href="#LineageHTML.__init__-269"><span class="linenos">269</span></a>
</span><span id="LineageHTML.__init__-270"><a href="#LineageHTML.__init__-270"><span class="linenos">270</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="p">:</span>
</span><span id="LineageHTML.__init__-271"><a href="#LineageHTML.__init__-271"><span class="linenos">271</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s2">&quot;from&quot;</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> <span class="s2">&quot;to&quot;</span><span class="p">:</span> <span class="nb">id</span><span class="p">(</span><span class="n">d</span><span class="p">)})</span>
</span></pre></div> </span></pre></div>

1144
docs/sqlglot/optimizer/annotate_types.html
View file

File diff suppressed because one or more lines are too long

2
docs/sqlglot/optimizer/merge_subqueries.html
View file

@ -568,7 +568,7 @@ queries if it would result in multiple table selects in a single query:</p>
<div class="attr variable"> <div class="attr variable">
<span class="name">UNMERGABLE_ARGS</span> = <span class="name">UNMERGABLE_ARGS</span> =
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;qualify&#39;, &#39;pivots&#39;, &#39;sample&#39;, &#39;locks&#39;, &#39;windows&#39;, &#39;distribute&#39;, &#39;laterals&#39;, &#39;sort&#39;, &#39;match&#39;, &#39;kind&#39;, &#39;settings&#39;, &#39;having&#39;, &#39;connect&#39;, &#39;format&#39;, &#39;into&#39;, &#39;offset&#39;, &#39;limit&#39;, &#39;distinct&#39;, &#39;with&#39;, &#39;cluster&#39;, &#39;group&#39;}</span> <label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{&#39;offset&#39;, &#39;connect&#39;, &#39;pivots&#39;, &#39;having&#39;, &#39;laterals&#39;, &#39;match&#39;, &#39;sample&#39;, &#39;kind&#39;, &#39;qualify&#39;, &#39;sort&#39;, &#39;limit&#39;, &#39;into&#39;, &#39;windows&#39;, &#39;locks&#39;, &#39;cluster&#39;, &#39;settings&#39;, &#39;with&#39;, &#39;distinct&#39;, &#39;group&#39;, &#39;distribute&#39;, &#39;format&#39;}</span>
</div> </div>

581
docs/sqlglot/optimizer/normalize.html
View file

@ -74,187 +74,184 @@
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a> </span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a>
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">exp</span> </span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">exp</span>
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot.errors</span> <span class="kn">import</span> <span class="n">OptimizeError</span> </span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot.errors</span> <span class="kn">import</span> <span class="n">OptimizeError</span>
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot.generator</span> <span class="kn">import</span> <span class="n">cached_generator</span> </span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot.helper</span> <span class="kn">import</span> <span class="n">while_changing</span>
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span> <span class="nn">sqlglot.helper</span> <span class="kn">import</span> <span class="n">while_changing</span> </span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="n">find_all_in_scope</span>
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="n">find_all_in_scope</span> </span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.simplify</span> <span class="kn">import</span> <span class="n">flatten</span><span class="p">,</span> <span class="n">rewrite_between</span><span class="p">,</span> <span class="n">uniq_sort</span>
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.simplify</span> <span class="kn">import</span> <span class="n">flatten</span><span class="p">,</span> <span class="n">rewrite_between</span><span class="p">,</span> <span class="n">uniq_sort</span> </span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a>
</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a> </span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s2">&quot;sqlglot&quot;</span><span class="p">)</span>
</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a><span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s2">&quot;sqlglot&quot;</span><span class="p">)</span> </span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a>
</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a> </span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a>
</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a> </span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a><span class="k">def</span> <span class="nf">normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">128</span><span class="p">):</span>
</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a><span class="k">def</span> <span class="nf">normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">128</span><span class="p">):</span> </span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="sd"> Rewrite sqlglot AST into conjunctive normal form or disjunctive normal form.</span>
</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a><span class="sd"> Rewrite sqlglot AST into conjunctive normal form or disjunctive normal form.</span> </span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a>
</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a> </span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a><span class="sd"> Example:</span>
</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a><span class="sd"> Example:</span> </span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span> </span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;(x AND y) OR z&quot;)</span>
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;(x AND y) OR z&quot;)</span> </span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a><span class="sd"> &gt;&gt;&gt; normalize(expression, dnf=False).sql()</span>
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a><span class="sd"> &gt;&gt;&gt; normalize(expression, dnf=False).sql()</span> </span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a><span class="sd"> &#39;(x OR z) AND (y OR z)&#39;</span>
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a><span class="sd"> &#39;(x OR z) AND (y OR z)&#39;</span> </span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a>
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a> </span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a><span class="sd"> Args:</span>
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="sd"> Args:</span> </span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="sd"> expression: expression to normalize</span>
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a><span class="sd"> expression: expression to normalize</span> </span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a><span class="sd"> dnf: rewrite in disjunctive normal form instead.</span>
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a><span class="sd"> dnf: rewrite in disjunctive normal form instead.</span> </span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a><span class="sd"> max_distance (int): the maximal estimated distance from cnf/dnf to attempt conversion</span>
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a><span class="sd"> max_distance (int): the maximal estimated distance from cnf/dnf to attempt conversion</span> </span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a><span class="sd"> Returns:</span>
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="sd"> Returns:</span> </span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="sd"> sqlglot.Expression: normalized expression</span>
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a><span class="sd"> sqlglot.Expression: normalized expression</span> </span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a> <span class="k">for</span> <span class="n">node</span><span class="p">,</span> <span class="o">*</span><span class="n">_</span> <span class="ow">in</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="o">*</span><span class="n">_</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">))):</span>
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a> <span class="n">generate</span> <span class="o">=</span> <span class="n">cached_generator</span><span class="p">()</span> </span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a> </span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a> <span class="k">if</span> <span class="n">normalized</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">):</span>
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a> <span class="k">for</span> <span class="n">node</span><span class="p">,</span> <span class="o">*</span><span class="n">_</span> <span class="ow">in</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="o">*</span><span class="n">_</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">))):</span> </span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a> <span class="k">continue</span>
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span> </span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a> <span class="n">root</span> <span class="o">=</span> <span class="n">node</span> <span class="ow">is</span> <span class="n">expression</span>
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a> <span class="k">if</span> <span class="n">normalized</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">):</span> </span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a> <span class="n">original</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a> <span class="k">continue</span> </span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a>
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a> <span class="n">root</span> <span class="o">=</span> <span class="n">node</span> <span class="ow">is</span> <span class="n">expression</span> </span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a> <span class="n">node</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">rewrite_between</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="n">original</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span> </span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a> <span class="n">distance</span> <span class="o">=</span> <span class="n">normalization_distance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">)</span>
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a> </span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a>
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="n">node</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">rewrite_between</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> </span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="k">if</span> <span class="n">distance</span> <span class="o">&gt;</span> <span class="n">max_distance</span><span class="p">:</span>
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="n">distance</span> <span class="o">=</span> <span class="n">normalization_distance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">)</span> </span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span>
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> </span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> <span class="sa">f</span><span class="s2">&quot;Skipping normalization because distance </span><span class="si">{</span><span class="n">distance</span><span class="si">}</span><span class="s2"> exceeds max </span><span class="si">{</span><span class="n">max_distance</span><span class="si">}</span><span class="s2">&quot;</span>
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="k">if</span> <span class="n">distance</span> <span class="o">&gt;</span> <span class="n">max_distance</span><span class="p">:</span> </span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="p">)</span>
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span> </span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="sa">f</span><span class="s2">&quot;Skipping normalization because distance </span><span class="si">{</span><span class="n">distance</span><span class="si">}</span><span class="s2"> exceeds max </span><span class="si">{</span><span class="n">max_distance</span><span class="si">}</span><span class="s2">&quot;</span> </span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a>
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="p">)</span> </span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="k">try</span><span class="p">:</span>
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span>
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> </span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">while_changing</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">e</span><span class="p">:</span> <span class="n">distributive_law</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">dnf</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">))</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="k">try</span><span class="p">:</span> </span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="p">)</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span> </span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="k">except</span> <span class="n">OptimizeError</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="n">while_changing</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">e</span><span class="p">:</span> <span class="n">distributive_law</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">dnf</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">,</span> <span class="n">generate</span><span class="p">))</span> </span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="p">)</span> </span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">original</span><span class="p">)</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="k">except</span> <span class="n">OptimizeError</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span> </span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="k">if</span> <span class="n">root</span><span class="p">:</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="n">e</span><span class="p">)</span> </span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="k">return</span> <span class="n">original</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">original</span><span class="p">)</span> </span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="k">if</span> <span class="n">root</span><span class="p">:</span> </span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a>
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="k">return</span> <span class="n">original</span> </span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="k">if</span> <span class="n">root</span><span class="p">:</span>
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">node</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> </span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a>
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="k">if</span> <span class="n">root</span><span class="p">:</span> </span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">node</span> </span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a>
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> </span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a>
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a><span class="k">def</span> <span class="nf">normalized</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">bool</span><span class="p">:</span>
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> </span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> </span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a><span class="sd"> Checks whether a given expression is in a normal form of interest.</span>
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a><span class="k">def</span> <span class="nf">normalized</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">bool</span><span class="p">:</span> </span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a>
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a><span class="sd"> Example:</span>
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a><span class="sd"> Checks whether a given expression is in a normal form of interest.</span> </span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a><span class="sd"> &gt;&gt;&gt; from sqlglot import parse_one</span>
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> </span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a><span class="sd"> &gt;&gt;&gt; normalized(parse_one(&quot;(a AND b) OR c OR (d AND e)&quot;), dnf=True)</span>
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a><span class="sd"> Example:</span> </span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a><span class="sd"> True</span>
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a><span class="sd"> &gt;&gt;&gt; from sqlglot import parse_one</span> </span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a><span class="sd"> &gt;&gt;&gt; normalized(parse_one(&quot;(a OR b) AND c&quot;)) # Checks CNF by default</span>
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a><span class="sd"> &gt;&gt;&gt; normalized(parse_one(&quot;(a AND b) OR c OR (d AND e)&quot;), dnf=True)</span> </span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a><span class="sd"> True</span>
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a><span class="sd"> True</span> </span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a><span class="sd"> &gt;&gt;&gt; normalized(parse_one(&quot;a AND (b OR c)&quot;), dnf=True)</span>
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a><span class="sd"> &gt;&gt;&gt; normalized(parse_one(&quot;(a OR b) AND c&quot;)) # Checks CNF by default</span> </span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a><span class="sd"> False</span>
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a><span class="sd"> True</span> </span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a>
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a><span class="sd"> &gt;&gt;&gt; normalized(parse_one(&quot;a AND (b OR c)&quot;), dnf=True)</span> </span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a><span class="sd"> Args:</span>
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a><span class="sd"> False</span> </span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a><span class="sd"> expression: The expression to check if it&#39;s normalized.</span>
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> </span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a><span class="sd"> dnf: Whether or not to check if the expression is in Disjunctive Normal Form (DNF).</span>
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a><span class="sd"> Args:</span> </span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a><span class="sd"> Default: False, i.e. we check if it&#39;s in Conjunctive Normal Form (CNF).</span>
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a><span class="sd"> expression: The expression to check if it&#39;s normalized.</span> </span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a><span class="sd"> dnf: Whether or not to check if the expression is in Disjunctive Normal Form (DNF).</span> </span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="n">ancestor</span><span class="p">,</span> <span class="n">root</span> <span class="o">=</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span> <span class="k">if</span> <span class="n">dnf</span> <span class="k">else</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">)</span>
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a><span class="sd"> Default: False, i.e. we check if it&#39;s in Conjunctive Normal Form (CNF).</span> </span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span>
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="n">connector</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">ancestor</span><span class="p">)</span> <span class="k">for</span> <span class="n">connector</span> <span class="ow">in</span> <span class="n">find_all_in_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">root</span><span class="p">)</span>
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="n">ancestor</span><span class="p">,</span> <span class="n">root</span> <span class="o">=</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span> <span class="k">if</span> <span class="n">dnf</span> <span class="k">else</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">)</span> </span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="p">)</span>
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span> </span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="n">connector</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">ancestor</span><span class="p">)</span> <span class="k">for</span> <span class="n">connector</span> <span class="ow">in</span> <span class="n">find_all_in_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">root</span><span class="p">)</span> </span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a>
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="p">)</span> </span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a><span class="k">def</span> <span class="nf">normalization_distance</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> </span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> </span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a><span class="sd"> The difference in the number of predicates between a given expression and its normalized form.</span>
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a><span class="k">def</span> <span class="nf">normalization_distance</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span> </span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a>
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a><span class="sd"> This is used as an estimate of the cost of the conversion which is exponential in complexity.</span>
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a><span class="sd"> The difference in the number of predicates between a given expression and its normalized form.</span> </span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a>
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> </span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a><span class="sd"> Example:</span>
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a><span class="sd"> This is used as an estimate of the cost of the conversion which is exponential in complexity.</span> </span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> </span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;(a AND b) OR (c AND d)&quot;)</span>
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a><span class="sd"> Example:</span> </span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a><span class="sd"> &gt;&gt;&gt; normalization_distance(expression)</span>
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span> </span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a><span class="sd"> 4</span>
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;(a AND b) OR (c AND d)&quot;)</span> </span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a>
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a><span class="sd"> &gt;&gt;&gt; normalization_distance(expression)</span> </span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a><span class="sd"> Args:</span>
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a><span class="sd"> 4</span> </span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a><span class="sd"> expression: The expression to compute the normalization distance for.</span>
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> </span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a><span class="sd"> dnf: Whether or not to check if the expression is in Disjunctive Normal Form (DNF).</span>
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a><span class="sd"> Args:</span> </span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a><span class="sd"> Default: False, i.e. we check if it&#39;s in Conjunctive Normal Form (CNF).</span>
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a><span class="sd"> expression: The expression to compute the normalization distance for.</span> </span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a><span class="sd"> dnf: Whether or not to check if the expression is in Disjunctive Normal Form (DNF).</span> </span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a><span class="sd"> Returns:</span>
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a><span class="sd"> Default: False, i.e. we check if it&#39;s in Conjunctive Normal Form (CNF).</span> </span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a><span class="sd"> The normalization distance.</span>
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> </span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a><span class="sd"> Returns:</span> </span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="k">return</span> <span class="nb">sum</span><span class="p">(</span><span class="n">_predicate_lengths</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">))</span> <span class="o">-</span> <span class="p">(</span>
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a><span class="sd"> The normalization distance.</span> </span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="nb">sum</span><span class="p">(</span><span class="mi">1</span> <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">))</span> <span class="o">+</span> <span class="mi">1</span>
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="p">)</span>
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">return</span> <span class="nb">sum</span><span class="p">(</span><span class="n">_predicate_lengths</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">))</span> <span class="o">-</span> <span class="p">(</span> </span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a>
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="nb">sum</span><span class="p">(</span><span class="mi">1</span> <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">))</span> <span class="o">+</span> <span class="mi">1</span> </span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a>
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="p">)</span> </span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a><span class="k">def</span> <span class="nf">_predicate_lengths</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">):</span>
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> </span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> </span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a><span class="sd"> Returns a list of predicate lengths when expanded to normalized form.</span>
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a><span class="k">def</span> <span class="nf">_predicate_lengths</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">):</span> </span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a>
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a><span class="sd"> (A AND B) OR C -&gt; [2, 2] because len(A OR C), len(B OR C).</span>
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a><span class="sd"> Returns a list of predicate lengths when expanded to normalized form.</span> </span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> </span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a><span class="sd"> (A AND B) OR C -&gt; [2, 2] because len(A OR C), len(B OR C).</span> </span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a>
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span> </span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="k">return</span> <span class="p">(</span><span class="mi">1</span><span class="p">,)</span>
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> </span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a>
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span> </span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">left</span><span class="p">,</span> <span class="n">right</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">values</span><span class="p">()</span>
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="k">return</span> <span class="p">(</span><span class="mi">1</span><span class="p">,)</span> </span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a>
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> </span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">if</span> <span class="n">dnf</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">):</span>
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">left</span><span class="p">,</span> <span class="n">right</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">values</span><span class="p">()</span> </span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="k">return</span> <span class="nb">tuple</span><span class="p">(</span>
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> </span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">a</span> <span class="o">+</span> <span class="n">b</span> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">_predicate_lengths</span><span class="p">(</span><span class="n">left</span><span class="p">,</span> <span class="n">dnf</span><span class="p">)</span> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">_predicate_lengths</span><span class="p">(</span><span class="n">right</span><span class="p">,</span> <span class="n">dnf</span><span class="p">)</span>
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">if</span> <span class="n">dnf</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">):</span> </span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="p">)</span>
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">return</span> <span class="nb">tuple</span><span class="p">(</span> </span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">return</span> <span class="n">_predicate_lengths</span><span class="p">(</span><span class="n">left</span><span class="p">,</span> <span class="n">dnf</span><span class="p">)</span> <span class="o">+</span> <span class="n">_predicate_lengths</span><span class="p">(</span><span class="n">right</span><span class="p">,</span> <span class="n">dnf</span><span class="p">)</span>
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">a</span> <span class="o">+</span> <span class="n">b</span> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">_predicate_lengths</span><span class="p">(</span><span class="n">left</span><span class="p">,</span> <span class="n">dnf</span><span class="p">)</span> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">_predicate_lengths</span><span class="p">(</span><span class="n">right</span><span class="p">,</span> <span class="n">dnf</span><span class="p">)</span> </span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a>
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="p">)</span> </span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a>
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">return</span> <span class="n">_predicate_lengths</span><span class="p">(</span><span class="n">left</span><span class="p">,</span> <span class="n">dnf</span><span class="p">)</span> <span class="o">+</span> <span class="n">_predicate_lengths</span><span class="p">(</span><span class="n">right</span><span class="p">,</span> <span class="n">dnf</span><span class="p">)</span> </span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a><span class="k">def</span> <span class="nf">distributive_law</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">):</span>
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> </span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> </span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a><span class="sd"> x OR (y AND z) -&gt; (x OR y) AND (x OR z)</span>
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a><span class="k">def</span> <span class="nf">distributive_law</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">,</span> <span class="n">generate</span><span class="p">):</span> </span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a><span class="sd"> (x AND y) OR (y AND z) -&gt; (x OR y) AND (x OR z) AND (y OR y) AND (y OR z)</span>
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a><span class="sd"> x OR (y AND z) -&gt; (x OR y) AND (x OR z)</span> </span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="k">if</span> <span class="n">normalized</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">):</span>
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a><span class="sd"> (x AND y) OR (y AND z) -&gt; (x OR y) AND (x OR z) AND (y OR y) AND (y OR z)</span> </span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a>
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="k">if</span> <span class="n">normalized</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">):</span> </span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">distance</span> <span class="o">=</span> <span class="n">normalization_distance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">)</span>
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a>
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> </span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="n">distance</span> <span class="o">&gt;</span> <span class="n">max_distance</span><span class="p">:</span>
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="n">distance</span> <span class="o">=</span> <span class="n">normalization_distance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">)</span> </span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;Normalization distance </span><span class="si">{</span><span class="n">distance</span><span class="si">}</span><span class="s2"> exceeds max </span><span class="si">{</span><span class="n">max_distance</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">)</span>
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> </span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a>
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="k">if</span> <span class="n">distance</span> <span class="o">&gt;</span> <span class="n">max_distance</span><span class="p">:</span> </span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">e</span><span class="p">:</span> <span class="n">distributive_law</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">dnf</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">))</span>
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;Normalization distance </span><span class="si">{</span><span class="n">distance</span><span class="si">}</span><span class="s2"> exceeds max </span><span class="si">{</span><span class="n">max_distance</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">)</span> </span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">to_exp</span><span class="p">,</span> <span class="n">from_exp</span> <span class="o">=</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">)</span> <span class="k">if</span> <span class="n">dnf</span> <span class="k">else</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span>
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> </span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a>
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">e</span><span class="p">:</span> <span class="n">distributive_law</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">dnf</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">,</span> <span class="n">generate</span><span class="p">))</span> </span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">from_exp</span><span class="p">):</span>
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">to_exp</span><span class="p">,</span> <span class="n">from_exp</span> <span class="o">=</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">)</span> <span class="k">if</span> <span class="n">dnf</span> <span class="k">else</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span> </span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">unnest_operands</span><span class="p">()</span>
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> </span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a>
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">from_exp</span><span class="p">):</span> </span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="n">from_func</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span> <span class="k">if</span> <span class="n">from_exp</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span>
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">unnest_operands</span><span class="p">()</span> </span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="n">to_func</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span> <span class="k">if</span> <span class="n">to_exp</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span>
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> </span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a>
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="n">from_func</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span> <span class="k">if</span> <span class="n">from_exp</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span> </span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">):</span>
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="n">to_func</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span> <span class="k">if</span> <span class="n">to_exp</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span> </span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="nb">tuple</span><span class="p">(</span><span class="n">a</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">)))</span> <span class="o">&gt;</span> <span class="nb">len</span><span class="p">(</span><span class="nb">tuple</span><span class="p">(</span><span class="n">b</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">))):</span>
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> </span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">)</span>
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">):</span> </span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">)</span>
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="nb">tuple</span><span class="p">(</span><span class="n">a</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">)))</span> <span class="o">&gt;</span> <span class="nb">len</span><span class="p">(</span><span class="nb">tuple</span><span class="p">(</span><span class="n">b</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">))):</span> </span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">):</span>
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">,</span> <span class="n">generate</span><span class="p">)</span> </span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">)</span>
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">,</span> <span class="n">generate</span><span class="p">)</span> </span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">):</span>
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">):</span> </span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">)</span>
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">,</span> <span class="n">generate</span><span class="p">)</span> </span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a>
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">):</span> </span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">,</span> <span class="n">generate</span><span class="p">)</span> </span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a>
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> </span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a>
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a><span class="k">def</span> <span class="nf">_distribute</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">):</span>
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> </span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> </span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span>
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a><span class="k">def</span> <span class="nf">_distribute</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">,</span> <span class="n">generate</span><span class="p">):</span> </span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="n">a</span><span class="p">,</span>
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span> </span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="k">lambda</span> <span class="n">c</span><span class="p">:</span> <span class="n">to_func</span><span class="p">(</span>
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span> </span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="n">uniq_sort</span><span class="p">(</span><span class="n">flatten</span><span class="p">(</span><span class="n">from_func</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">left</span><span class="p">))),</span>
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="n">a</span><span class="p">,</span> </span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="n">uniq_sort</span><span class="p">(</span><span class="n">flatten</span><span class="p">(</span><span class="n">from_func</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">right</span><span class="p">))),</span>
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="k">lambda</span> <span class="n">c</span><span class="p">:</span> <span class="n">to_func</span><span class="p">(</span> </span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="n">uniq_sort</span><span class="p">(</span><span class="n">flatten</span><span class="p">(</span><span class="n">from_func</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">left</span><span class="p">)),</span> <span class="n">generate</span><span class="p">),</span> </span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="p">),</span>
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="n">uniq_sort</span><span class="p">(</span><span class="n">flatten</span><span class="p">(</span><span class="n">from_func</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">right</span><span class="p">)),</span> <span class="n">generate</span><span class="p">),</span> </span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="p">)</span>
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> </span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="k">else</span><span class="p">:</span>
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="p">),</span> </span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="n">a</span> <span class="o">=</span> <span class="n">to_func</span><span class="p">(</span>
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="p">)</span> </span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="n">uniq_sort</span><span class="p">(</span><span class="n">flatten</span><span class="p">(</span><span class="n">from_func</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">left</span><span class="p">))),</span>
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="k">else</span><span class="p">:</span> </span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="n">uniq_sort</span><span class="p">(</span><span class="n">flatten</span><span class="p">(</span><span class="n">from_func</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">right</span><span class="p">))),</span>
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="n">a</span> <span class="o">=</span> <span class="n">to_func</span><span class="p">(</span> </span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="n">uniq_sort</span><span class="p">(</span><span class="n">flatten</span><span class="p">(</span><span class="n">from_func</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">left</span><span class="p">)),</span> <span class="n">generate</span><span class="p">),</span> </span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="p">)</span>
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="n">uniq_sort</span><span class="p">(</span><span class="n">flatten</span><span class="p">(</span><span class="n">from_func</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">.</span><span class="n">right</span><span class="p">)),</span> <span class="n">generate</span><span class="p">),</span> </span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a>
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> </span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="k">return</span> <span class="n">a</span>
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="p">)</span>
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a>
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="k">return</span> <span class="n">a</span>
</span></pre></div> </span></pre></div>
@ -282,56 +279,54 @@
</div> </div>
<a class="headerlink" href="#normalize"></a> <a class="headerlink" href="#normalize"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="normalize-16"><a href="#normalize-16"><span class="linenos">16</span></a><span class="k">def</span> <span class="nf">normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">128</span><span class="p">):</span> <div class="pdoc-code codehilite"><pre><span></span><span id="normalize-15"><a href="#normalize-15"><span class="linenos">15</span></a><span class="k">def</span> <span class="nf">normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">128</span><span class="p">):</span>
</span><span id="normalize-17"><a href="#normalize-17"><span class="linenos">17</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="normalize-16"><a href="#normalize-16"><span class="linenos">16</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="normalize-18"><a href="#normalize-18"><span class="linenos">18</span></a><span class="sd"> Rewrite sqlglot AST into conjunctive normal form or disjunctive normal form.</span> </span><span id="normalize-17"><a href="#normalize-17"><span class="linenos">17</span></a><span class="sd"> Rewrite sqlglot AST into conjunctive normal form or disjunctive normal form.</span>
</span><span id="normalize-19"><a href="#normalize-19"><span class="linenos">19</span></a> </span><span id="normalize-18"><a href="#normalize-18"><span class="linenos">18</span></a>
</span><span id="normalize-20"><a href="#normalize-20"><span class="linenos">20</span></a><span class="sd"> Example:</span> </span><span id="normalize-19"><a href="#normalize-19"><span class="linenos">19</span></a><span class="sd"> Example:</span>
</span><span id="normalize-21"><a href="#normalize-21"><span class="linenos">21</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span> </span><span id="normalize-20"><a href="#normalize-20"><span class="linenos">20</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="normalize-22"><a href="#normalize-22"><span class="linenos">22</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;(x AND y) OR z&quot;)</span> </span><span id="normalize-21"><a href="#normalize-21"><span class="linenos">21</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;(x AND y) OR z&quot;)</span>
</span><span id="normalize-23"><a href="#normalize-23"><span class="linenos">23</span></a><span class="sd"> &gt;&gt;&gt; normalize(expression, dnf=False).sql()</span> </span><span id="normalize-22"><a href="#normalize-22"><span class="linenos">22</span></a><span class="sd"> &gt;&gt;&gt; normalize(expression, dnf=False).sql()</span>
</span><span id="normalize-24"><a href="#normalize-24"><span class="linenos">24</span></a><span class="sd"> &#39;(x OR z) AND (y OR z)&#39;</span> </span><span id="normalize-23"><a href="#normalize-23"><span class="linenos">23</span></a><span class="sd"> &#39;(x OR z) AND (y OR z)&#39;</span>
</span><span id="normalize-25"><a href="#normalize-25"><span class="linenos">25</span></a> </span><span id="normalize-24"><a href="#normalize-24"><span class="linenos">24</span></a>
</span><span id="normalize-26"><a href="#normalize-26"><span class="linenos">26</span></a><span class="sd"> Args:</span> </span><span id="normalize-25"><a href="#normalize-25"><span class="linenos">25</span></a><span class="sd"> Args:</span>
</span><span id="normalize-27"><a href="#normalize-27"><span class="linenos">27</span></a><span class="sd"> expression: expression to normalize</span> </span><span id="normalize-26"><a href="#normalize-26"><span class="linenos">26</span></a><span class="sd"> expression: expression to normalize</span>
</span><span id="normalize-28"><a href="#normalize-28"><span class="linenos">28</span></a><span class="sd"> dnf: rewrite in disjunctive normal form instead.</span> </span><span id="normalize-27"><a href="#normalize-27"><span class="linenos">27</span></a><span class="sd"> dnf: rewrite in disjunctive normal form instead.</span>
</span><span id="normalize-29"><a href="#normalize-29"><span class="linenos">29</span></a><span class="sd"> max_distance (int): the maximal estimated distance from cnf/dnf to attempt conversion</span> </span><span id="normalize-28"><a href="#normalize-28"><span class="linenos">28</span></a><span class="sd"> max_distance (int): the maximal estimated distance from cnf/dnf to attempt conversion</span>
</span><span id="normalize-30"><a href="#normalize-30"><span class="linenos">30</span></a><span class="sd"> Returns:</span> </span><span id="normalize-29"><a href="#normalize-29"><span class="linenos">29</span></a><span class="sd"> Returns:</span>
</span><span id="normalize-31"><a href="#normalize-31"><span class="linenos">31</span></a><span class="sd"> sqlglot.Expression: normalized expression</span> </span><span id="normalize-30"><a href="#normalize-30"><span class="linenos">30</span></a><span class="sd"> sqlglot.Expression: normalized expression</span>
</span><span id="normalize-32"><a href="#normalize-32"><span class="linenos">32</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="normalize-31"><a href="#normalize-31"><span class="linenos">31</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="normalize-33"><a href="#normalize-33"><span class="linenos">33</span></a> <span class="n">generate</span> <span class="o">=</span> <span class="n">cached_generator</span><span class="p">()</span> </span><span id="normalize-32"><a href="#normalize-32"><span class="linenos">32</span></a> <span class="k">for</span> <span class="n">node</span><span class="p">,</span> <span class="o">*</span><span class="n">_</span> <span class="ow">in</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="o">*</span><span class="n">_</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">))):</span>
</span><span id="normalize-34"><a href="#normalize-34"><span class="linenos">34</span></a> </span><span id="normalize-33"><a href="#normalize-33"><span class="linenos">33</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
</span><span id="normalize-35"><a href="#normalize-35"><span class="linenos">35</span></a> <span class="k">for</span> <span class="n">node</span><span class="p">,</span> <span class="o">*</span><span class="n">_</span> <span class="ow">in</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="n">e</span><span class="p">,</span> <span class="o">*</span><span class="n">_</span><span class="p">:</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">))):</span> </span><span id="normalize-34"><a href="#normalize-34"><span class="linenos">34</span></a> <span class="k">if</span> <span class="n">normalized</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">):</span>
</span><span id="normalize-36"><a href="#normalize-36"><span class="linenos">36</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span> </span><span id="normalize-35"><a href="#normalize-35"><span class="linenos">35</span></a> <span class="k">continue</span>
</span><span id="normalize-37"><a href="#normalize-37"><span class="linenos">37</span></a> <span class="k">if</span> <span class="n">normalized</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">):</span> </span><span id="normalize-36"><a href="#normalize-36"><span class="linenos">36</span></a> <span class="n">root</span> <span class="o">=</span> <span class="n">node</span> <span class="ow">is</span> <span class="n">expression</span>
</span><span id="normalize-38"><a href="#normalize-38"><span class="linenos">38</span></a> <span class="k">continue</span> </span><span id="normalize-37"><a href="#normalize-37"><span class="linenos">37</span></a> <span class="n">original</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
</span><span id="normalize-39"><a href="#normalize-39"><span class="linenos">39</span></a> <span class="n">root</span> <span class="o">=</span> <span class="n">node</span> <span class="ow">is</span> <span class="n">expression</span> </span><span id="normalize-38"><a href="#normalize-38"><span class="linenos">38</span></a>
</span><span id="normalize-40"><a href="#normalize-40"><span class="linenos">40</span></a> <span class="n">original</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span> </span><span id="normalize-39"><a href="#normalize-39"><span class="linenos">39</span></a> <span class="n">node</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">rewrite_between</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
</span><span id="normalize-40"><a href="#normalize-40"><span class="linenos">40</span></a> <span class="n">distance</span> <span class="o">=</span> <span class="n">normalization_distance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">)</span>
</span><span id="normalize-41"><a href="#normalize-41"><span class="linenos">41</span></a> </span><span id="normalize-41"><a href="#normalize-41"><span class="linenos">41</span></a>
</span><span id="normalize-42"><a href="#normalize-42"><span class="linenos">42</span></a> <span class="n">node</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">rewrite_between</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> </span><span id="normalize-42"><a href="#normalize-42"><span class="linenos">42</span></a> <span class="k">if</span> <span class="n">distance</span> <span class="o">&gt;</span> <span class="n">max_distance</span><span class="p">:</span>
</span><span id="normalize-43"><a href="#normalize-43"><span class="linenos">43</span></a> <span class="n">distance</span> <span class="o">=</span> <span class="n">normalization_distance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">)</span> </span><span id="normalize-43"><a href="#normalize-43"><span class="linenos">43</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span>
</span><span id="normalize-44"><a href="#normalize-44"><span class="linenos">44</span></a> </span><span id="normalize-44"><a href="#normalize-44"><span class="linenos">44</span></a> <span class="sa">f</span><span class="s2">&quot;Skipping normalization because distance </span><span class="si">{</span><span class="n">distance</span><span class="si">}</span><span class="s2"> exceeds max </span><span class="si">{</span><span class="n">max_distance</span><span class="si">}</span><span class="s2">&quot;</span>
</span><span id="normalize-45"><a href="#normalize-45"><span class="linenos">45</span></a> <span class="k">if</span> <span class="n">distance</span> <span class="o">&gt;</span> <span class="n">max_distance</span><span class="p">:</span> </span><span id="normalize-45"><a href="#normalize-45"><span class="linenos">45</span></a> <span class="p">)</span>
</span><span id="normalize-46"><a href="#normalize-46"><span class="linenos">46</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span> </span><span id="normalize-46"><a href="#normalize-46"><span class="linenos">46</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="normalize-47"><a href="#normalize-47"><span class="linenos">47</span></a> <span class="sa">f</span><span class="s2">&quot;Skipping normalization because distance </span><span class="si">{</span><span class="n">distance</span><span class="si">}</span><span class="s2"> exceeds max </span><span class="si">{</span><span class="n">max_distance</span><span class="si">}</span><span class="s2">&quot;</span> </span><span id="normalize-47"><a href="#normalize-47"><span class="linenos">47</span></a>
</span><span id="normalize-48"><a href="#normalize-48"><span class="linenos">48</span></a> <span class="p">)</span> </span><span id="normalize-48"><a href="#normalize-48"><span class="linenos">48</span></a> <span class="k">try</span><span class="p">:</span>
</span><span id="normalize-49"><a href="#normalize-49"><span class="linenos">49</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="normalize-49"><a href="#normalize-49"><span class="linenos">49</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span>
</span><span id="normalize-50"><a href="#normalize-50"><span class="linenos">50</span></a> </span><span id="normalize-50"><a href="#normalize-50"><span class="linenos">50</span></a> <span class="n">while_changing</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">e</span><span class="p">:</span> <span class="n">distributive_law</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">dnf</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">))</span>
</span><span id="normalize-51"><a href="#normalize-51"><span class="linenos">51</span></a> <span class="k">try</span><span class="p">:</span> </span><span id="normalize-51"><a href="#normalize-51"><span class="linenos">51</span></a> <span class="p">)</span>
</span><span id="normalize-52"><a href="#normalize-52"><span class="linenos">52</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span> </span><span id="normalize-52"><a href="#normalize-52"><span class="linenos">52</span></a> <span class="k">except</span> <span class="n">OptimizeError</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
</span><span id="normalize-53"><a href="#normalize-53"><span class="linenos">53</span></a> <span class="n">while_changing</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">e</span><span class="p">:</span> <span class="n">distributive_law</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">dnf</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">,</span> <span class="n">generate</span><span class="p">))</span> </span><span id="normalize-53"><a href="#normalize-53"><span class="linenos">53</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
</span><span id="normalize-54"><a href="#normalize-54"><span class="linenos">54</span></a> <span class="p">)</span> </span><span id="normalize-54"><a href="#normalize-54"><span class="linenos">54</span></a> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">original</span><span class="p">)</span>
</span><span id="normalize-55"><a href="#normalize-55"><span class="linenos">55</span></a> <span class="k">except</span> <span class="n">OptimizeError</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span> </span><span id="normalize-55"><a href="#normalize-55"><span class="linenos">55</span></a> <span class="k">if</span> <span class="n">root</span><span class="p">:</span>
</span><span id="normalize-56"><a href="#normalize-56"><span class="linenos">56</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="n">e</span><span class="p">)</span> </span><span id="normalize-56"><a href="#normalize-56"><span class="linenos">56</span></a> <span class="k">return</span> <span class="n">original</span>
</span><span id="normalize-57"><a href="#normalize-57"><span class="linenos">57</span></a> <span class="n">node</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">original</span><span class="p">)</span> </span><span id="normalize-57"><a href="#normalize-57"><span class="linenos">57</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="normalize-58"><a href="#normalize-58"><span class="linenos">58</span></a> <span class="k">if</span> <span class="n">root</span><span class="p">:</span> </span><span id="normalize-58"><a href="#normalize-58"><span class="linenos">58</span></a>
</span><span id="normalize-59"><a href="#normalize-59"><span class="linenos">59</span></a> <span class="k">return</span> <span class="n">original</span> </span><span id="normalize-59"><a href="#normalize-59"><span class="linenos">59</span></a> <span class="k">if</span> <span class="n">root</span><span class="p">:</span>
</span><span id="normalize-60"><a href="#normalize-60"><span class="linenos">60</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="normalize-60"><a href="#normalize-60"><span class="linenos">60</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">node</span>
</span><span id="normalize-61"><a href="#normalize-61"><span class="linenos">61</span></a> </span><span id="normalize-61"><a href="#normalize-61"><span class="linenos">61</span></a>
</span><span id="normalize-62"><a href="#normalize-62"><span class="linenos">62</span></a> <span class="k">if</span> <span class="n">root</span><span class="p">:</span> </span><span id="normalize-62"><a href="#normalize-62"><span class="linenos">62</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="normalize-63"><a href="#normalize-63"><span class="linenos">63</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">node</span>
</span><span id="normalize-64"><a href="#normalize-64"><span class="linenos">64</span></a>
</span><span id="normalize-65"><a href="#normalize-65"><span class="linenos">65</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>
@ -377,28 +372,28 @@
</div> </div>
<a class="headerlink" href="#normalized"></a> <a class="headerlink" href="#normalized"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="normalized-68"><a href="#normalized-68"><span class="linenos">68</span></a><span class="k">def</span> <span class="nf">normalized</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">bool</span><span class="p">:</span> <div class="pdoc-code codehilite"><pre><span></span><span id="normalized-65"><a href="#normalized-65"><span class="linenos">65</span></a><span class="k">def</span> <span class="nf">normalized</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">bool</span><span class="p">:</span>
</span><span id="normalized-69"><a href="#normalized-69"><span class="linenos">69</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="normalized-66"><a href="#normalized-66"><span class="linenos">66</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="normalized-70"><a href="#normalized-70"><span class="linenos">70</span></a><span class="sd"> Checks whether a given expression is in a normal form of interest.</span> </span><span id="normalized-67"><a href="#normalized-67"><span class="linenos">67</span></a><span class="sd"> Checks whether a given expression is in a normal form of interest.</span>
</span><span id="normalized-71"><a href="#normalized-71"><span class="linenos">71</span></a> </span><span id="normalized-68"><a href="#normalized-68"><span class="linenos">68</span></a>
</span><span id="normalized-72"><a href="#normalized-72"><span class="linenos">72</span></a><span class="sd"> Example:</span> </span><span id="normalized-69"><a href="#normalized-69"><span class="linenos">69</span></a><span class="sd"> Example:</span>
</span><span id="normalized-73"><a href="#normalized-73"><span class="linenos">73</span></a><span class="sd"> &gt;&gt;&gt; from sqlglot import parse_one</span> </span><span id="normalized-70"><a href="#normalized-70"><span class="linenos">70</span></a><span class="sd"> &gt;&gt;&gt; from sqlglot import parse_one</span>
</span><span id="normalized-74"><a href="#normalized-74"><span class="linenos">74</span></a><span class="sd"> &gt;&gt;&gt; normalized(parse_one(&quot;(a AND b) OR c OR (d AND e)&quot;), dnf=True)</span> </span><span id="normalized-71"><a href="#normalized-71"><span class="linenos">71</span></a><span class="sd"> &gt;&gt;&gt; normalized(parse_one(&quot;(a AND b) OR c OR (d AND e)&quot;), dnf=True)</span>
</span><span id="normalized-75"><a href="#normalized-75"><span class="linenos">75</span></a><span class="sd"> True</span> </span><span id="normalized-72"><a href="#normalized-72"><span class="linenos">72</span></a><span class="sd"> True</span>
</span><span id="normalized-76"><a href="#normalized-76"><span class="linenos">76</span></a><span class="sd"> &gt;&gt;&gt; normalized(parse_one(&quot;(a OR b) AND c&quot;)) # Checks CNF by default</span> </span><span id="normalized-73"><a href="#normalized-73"><span class="linenos">73</span></a><span class="sd"> &gt;&gt;&gt; normalized(parse_one(&quot;(a OR b) AND c&quot;)) # Checks CNF by default</span>
</span><span id="normalized-77"><a href="#normalized-77"><span class="linenos">77</span></a><span class="sd"> True</span> </span><span id="normalized-74"><a href="#normalized-74"><span class="linenos">74</span></a><span class="sd"> True</span>
</span><span id="normalized-78"><a href="#normalized-78"><span class="linenos">78</span></a><span class="sd"> &gt;&gt;&gt; normalized(parse_one(&quot;a AND (b OR c)&quot;), dnf=True)</span> </span><span id="normalized-75"><a href="#normalized-75"><span class="linenos">75</span></a><span class="sd"> &gt;&gt;&gt; normalized(parse_one(&quot;a AND (b OR c)&quot;), dnf=True)</span>
</span><span id="normalized-79"><a href="#normalized-79"><span class="linenos">79</span></a><span class="sd"> False</span> </span><span id="normalized-76"><a href="#normalized-76"><span class="linenos">76</span></a><span class="sd"> False</span>
</span><span id="normalized-80"><a href="#normalized-80"><span class="linenos">80</span></a> </span><span id="normalized-77"><a href="#normalized-77"><span class="linenos">77</span></a>
</span><span id="normalized-81"><a href="#normalized-81"><span class="linenos">81</span></a><span class="sd"> Args:</span> </span><span id="normalized-78"><a href="#normalized-78"><span class="linenos">78</span></a><span class="sd"> Args:</span>
</span><span id="normalized-82"><a href="#normalized-82"><span class="linenos">82</span></a><span class="sd"> expression: The expression to check if it&#39;s normalized.</span> </span><span id="normalized-79"><a href="#normalized-79"><span class="linenos">79</span></a><span class="sd"> expression: The expression to check if it&#39;s normalized.</span>
</span><span id="normalized-83"><a href="#normalized-83"><span class="linenos">83</span></a><span class="sd"> dnf: Whether or not to check if the expression is in Disjunctive Normal Form (DNF).</span> </span><span id="normalized-80"><a href="#normalized-80"><span class="linenos">80</span></a><span class="sd"> dnf: Whether or not to check if the expression is in Disjunctive Normal Form (DNF).</span>
</span><span id="normalized-84"><a href="#normalized-84"><span class="linenos">84</span></a><span class="sd"> Default: False, i.e. we check if it&#39;s in Conjunctive Normal Form (CNF).</span> </span><span id="normalized-81"><a href="#normalized-81"><span class="linenos">81</span></a><span class="sd"> Default: False, i.e. we check if it&#39;s in Conjunctive Normal Form (CNF).</span>
</span><span id="normalized-85"><a href="#normalized-85"><span class="linenos">85</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="normalized-82"><a href="#normalized-82"><span class="linenos">82</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="normalized-86"><a href="#normalized-86"><span class="linenos">86</span></a> <span class="n">ancestor</span><span class="p">,</span> <span class="n">root</span> <span class="o">=</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span> <span class="k">if</span> <span class="n">dnf</span> <span class="k">else</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">)</span> </span><span id="normalized-83"><a href="#normalized-83"><span class="linenos">83</span></a> <span class="n">ancestor</span><span class="p">,</span> <span class="n">root</span> <span class="o">=</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span> <span class="k">if</span> <span class="n">dnf</span> <span class="k">else</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">)</span>
</span><span id="normalized-87"><a href="#normalized-87"><span class="linenos">87</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span> </span><span id="normalized-84"><a href="#normalized-84"><span class="linenos">84</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span>
</span><span id="normalized-88"><a href="#normalized-88"><span class="linenos">88</span></a> <span class="n">connector</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">ancestor</span><span class="p">)</span> <span class="k">for</span> <span class="n">connector</span> <span class="ow">in</span> <span class="n">find_all_in_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">root</span><span class="p">)</span> </span><span id="normalized-85"><a href="#normalized-85"><span class="linenos">85</span></a> <span class="n">connector</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">ancestor</span><span class="p">)</span> <span class="k">for</span> <span class="n">connector</span> <span class="ow">in</span> <span class="n">find_all_in_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">root</span><span class="p">)</span>
</span><span id="normalized-89"><a href="#normalized-89"><span class="linenos">89</span></a> <span class="p">)</span> </span><span id="normalized-86"><a href="#normalized-86"><span class="linenos">86</span></a> <span class="p">)</span>
</span></pre></div> </span></pre></div>
@ -441,29 +436,29 @@ Default: False, i.e. we check if it's in Conjunctive Normal Form (CNF).</li>
</div> </div>
<a class="headerlink" href="#normalization_distance"></a> <a class="headerlink" href="#normalization_distance"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="normalization_distance-92"><a href="#normalization_distance-92"><span class="linenos"> 92</span></a><span class="k">def</span> <span class="nf">normalization_distance</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span> <div class="pdoc-code codehilite"><pre><span></span><span id="normalization_distance-89"><a href="#normalization_distance-89"><span class="linenos"> 89</span></a><span class="k">def</span> <span class="nf">normalization_distance</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
</span><span id="normalization_distance-93"><a href="#normalization_distance-93"><span class="linenos"> 93</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="normalization_distance-90"><a href="#normalization_distance-90"><span class="linenos"> 90</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="normalization_distance-94"><a href="#normalization_distance-94"><span class="linenos"> 94</span></a><span class="sd"> The difference in the number of predicates between a given expression and its normalized form.</span> </span><span id="normalization_distance-91"><a href="#normalization_distance-91"><span class="linenos"> 91</span></a><span class="sd"> The difference in the number of predicates between a given expression and its normalized form.</span>
</span><span id="normalization_distance-95"><a href="#normalization_distance-95"><span class="linenos"> 95</span></a> </span><span id="normalization_distance-92"><a href="#normalization_distance-92"><span class="linenos"> 92</span></a>
</span><span id="normalization_distance-96"><a href="#normalization_distance-96"><span class="linenos"> 96</span></a><span class="sd"> This is used as an estimate of the cost of the conversion which is exponential in complexity.</span> </span><span id="normalization_distance-93"><a href="#normalization_distance-93"><span class="linenos"> 93</span></a><span class="sd"> This is used as an estimate of the cost of the conversion which is exponential in complexity.</span>
</span><span id="normalization_distance-97"><a href="#normalization_distance-97"><span class="linenos"> 97</span></a> </span><span id="normalization_distance-94"><a href="#normalization_distance-94"><span class="linenos"> 94</span></a>
</span><span id="normalization_distance-98"><a href="#normalization_distance-98"><span class="linenos"> 98</span></a><span class="sd"> Example:</span> </span><span id="normalization_distance-95"><a href="#normalization_distance-95"><span class="linenos"> 95</span></a><span class="sd"> Example:</span>
</span><span id="normalization_distance-99"><a href="#normalization_distance-99"><span class="linenos"> 99</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span> </span><span id="normalization_distance-96"><a href="#normalization_distance-96"><span class="linenos"> 96</span></a><span class="sd"> &gt;&gt;&gt; import sqlglot</span>
</span><span id="normalization_distance-100"><a href="#normalization_distance-100"><span class="linenos">100</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;(a AND b) OR (c AND d)&quot;)</span> </span><span id="normalization_distance-97"><a href="#normalization_distance-97"><span class="linenos"> 97</span></a><span class="sd"> &gt;&gt;&gt; expression = sqlglot.parse_one(&quot;(a AND b) OR (c AND d)&quot;)</span>
</span><span id="normalization_distance-101"><a href="#normalization_distance-101"><span class="linenos">101</span></a><span class="sd"> &gt;&gt;&gt; normalization_distance(expression)</span> </span><span id="normalization_distance-98"><a href="#normalization_distance-98"><span class="linenos"> 98</span></a><span class="sd"> &gt;&gt;&gt; normalization_distance(expression)</span>
</span><span id="normalization_distance-102"><a href="#normalization_distance-102"><span class="linenos">102</span></a><span class="sd"> 4</span> </span><span id="normalization_distance-99"><a href="#normalization_distance-99"><span class="linenos"> 99</span></a><span class="sd"> 4</span>
</span><span id="normalization_distance-103"><a href="#normalization_distance-103"><span class="linenos">103</span></a> </span><span id="normalization_distance-100"><a href="#normalization_distance-100"><span class="linenos">100</span></a>
</span><span id="normalization_distance-104"><a href="#normalization_distance-104"><span class="linenos">104</span></a><span class="sd"> Args:</span> </span><span id="normalization_distance-101"><a href="#normalization_distance-101"><span class="linenos">101</span></a><span class="sd"> Args:</span>
</span><span id="normalization_distance-105"><a href="#normalization_distance-105"><span class="linenos">105</span></a><span class="sd"> expression: The expression to compute the normalization distance for.</span> </span><span id="normalization_distance-102"><a href="#normalization_distance-102"><span class="linenos">102</span></a><span class="sd"> expression: The expression to compute the normalization distance for.</span>
</span><span id="normalization_distance-106"><a href="#normalization_distance-106"><span class="linenos">106</span></a><span class="sd"> dnf: Whether or not to check if the expression is in Disjunctive Normal Form (DNF).</span> </span><span id="normalization_distance-103"><a href="#normalization_distance-103"><span class="linenos">103</span></a><span class="sd"> dnf: Whether or not to check if the expression is in Disjunctive Normal Form (DNF).</span>
</span><span id="normalization_distance-107"><a href="#normalization_distance-107"><span class="linenos">107</span></a><span class="sd"> Default: False, i.e. we check if it&#39;s in Conjunctive Normal Form (CNF).</span> </span><span id="normalization_distance-104"><a href="#normalization_distance-104"><span class="linenos">104</span></a><span class="sd"> Default: False, i.e. we check if it&#39;s in Conjunctive Normal Form (CNF).</span>
</span><span id="normalization_distance-108"><a href="#normalization_distance-108"><span class="linenos">108</span></a> </span><span id="normalization_distance-105"><a href="#normalization_distance-105"><span class="linenos">105</span></a>
</span><span id="normalization_distance-109"><a href="#normalization_distance-109"><span class="linenos">109</span></a><span class="sd"> Returns:</span> </span><span id="normalization_distance-106"><a href="#normalization_distance-106"><span class="linenos">106</span></a><span class="sd"> Returns:</span>
</span><span id="normalization_distance-110"><a href="#normalization_distance-110"><span class="linenos">110</span></a><span class="sd"> The normalization distance.</span> </span><span id="normalization_distance-107"><a href="#normalization_distance-107"><span class="linenos">107</span></a><span class="sd"> The normalization distance.</span>
</span><span id="normalization_distance-111"><a href="#normalization_distance-111"><span class="linenos">111</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="normalization_distance-108"><a href="#normalization_distance-108"><span class="linenos">108</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="normalization_distance-112"><a href="#normalization_distance-112"><span class="linenos">112</span></a> <span class="k">return</span> <span class="nb">sum</span><span class="p">(</span><span class="n">_predicate_lengths</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">))</span> <span class="o">-</span> <span class="p">(</span> </span><span id="normalization_distance-109"><a href="#normalization_distance-109"><span class="linenos">109</span></a> <span class="k">return</span> <span class="nb">sum</span><span class="p">(</span><span class="n">_predicate_lengths</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">))</span> <span class="o">-</span> <span class="p">(</span>
</span><span id="normalization_distance-113"><a href="#normalization_distance-113"><span class="linenos">113</span></a> <span class="nb">sum</span><span class="p">(</span><span class="mi">1</span> <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">))</span> <span class="o">+</span> <span class="mi">1</span> </span><span id="normalization_distance-110"><a href="#normalization_distance-110"><span class="linenos">110</span></a> <span class="nb">sum</span><span class="p">(</span><span class="mi">1</span> <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">))</span> <span class="o">+</span> <span class="mi">1</span>
</span><span id="normalization_distance-114"><a href="#normalization_distance-114"><span class="linenos">114</span></a> <span class="p">)</span> </span><span id="normalization_distance-111"><a href="#normalization_distance-111"><span class="linenos">111</span></a> <span class="p">)</span>
</span></pre></div> </span></pre></div>
@ -505,44 +500,44 @@ Default: False, i.e. we check if it's in Conjunctive Normal Form (CNF).</li>
<div class="attr function"> <div class="attr function">
<span class="def">def</span> <span class="def">def</span>
<span class="name">distributive_law</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">expression</span>, </span><span class="param"><span class="n">dnf</span>, </span><span class="param"><span class="n">max_distance</span>, </span><span class="param"><span class="n">generate</span></span><span class="return-annotation">):</span></span> <span class="name">distributive_law</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">expression</span>, </span><span class="param"><span class="n">dnf</span>, </span><span class="param"><span class="n">max_distance</span></span><span class="return-annotation">):</span></span>
<label class="view-source-button" for="distributive_law-view-source"><span>View Source</span></label> <label class="view-source-button" for="distributive_law-view-source"><span>View Source</span></label>
</div> </div>
<a class="headerlink" href="#distributive_law"></a> <a class="headerlink" href="#distributive_law"></a>
<div class="pdoc-code codehilite"><pre><span></span><span id="distributive_law-137"><a href="#distributive_law-137"><span class="linenos">137</span></a><span class="k">def</span> <span class="nf">distributive_law</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">,</span> <span class="n">generate</span><span class="p">):</span> <div class="pdoc-code codehilite"><pre><span></span><span id="distributive_law-134"><a href="#distributive_law-134"><span class="linenos">134</span></a><span class="k">def</span> <span class="nf">distributive_law</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">):</span>
</span><span id="distributive_law-138"><a href="#distributive_law-138"><span class="linenos">138</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span> </span><span id="distributive_law-135"><a href="#distributive_law-135"><span class="linenos">135</span></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="distributive_law-139"><a href="#distributive_law-139"><span class="linenos">139</span></a><span class="sd"> x OR (y AND z) -&gt; (x OR y) AND (x OR z)</span> </span><span id="distributive_law-136"><a href="#distributive_law-136"><span class="linenos">136</span></a><span class="sd"> x OR (y AND z) -&gt; (x OR y) AND (x OR z)</span>
</span><span id="distributive_law-140"><a href="#distributive_law-140"><span class="linenos">140</span></a><span class="sd"> (x AND y) OR (y AND z) -&gt; (x OR y) AND (x OR z) AND (y OR y) AND (y OR z)</span> </span><span id="distributive_law-137"><a href="#distributive_law-137"><span class="linenos">137</span></a><span class="sd"> (x AND y) OR (y AND z) -&gt; (x OR y) AND (x OR z) AND (y OR y) AND (y OR z)</span>
</span><span id="distributive_law-141"><a href="#distributive_law-141"><span class="linenos">141</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="distributive_law-138"><a href="#distributive_law-138"><span class="linenos">138</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="distributive_law-142"><a href="#distributive_law-142"><span class="linenos">142</span></a> <span class="k">if</span> <span class="n">normalized</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">):</span> </span><span id="distributive_law-139"><a href="#distributive_law-139"><span class="linenos">139</span></a> <span class="k">if</span> <span class="n">normalized</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">):</span>
</span><span id="distributive_law-143"><a href="#distributive_law-143"><span class="linenos">143</span></a> <span class="k">return</span> <span class="n">expression</span> </span><span id="distributive_law-140"><a href="#distributive_law-140"><span class="linenos">140</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="distributive_law-144"><a href="#distributive_law-144"><span class="linenos">144</span></a> </span><span id="distributive_law-141"><a href="#distributive_law-141"><span class="linenos">141</span></a>
</span><span id="distributive_law-145"><a href="#distributive_law-145"><span class="linenos">145</span></a> <span class="n">distance</span> <span class="o">=</span> <span class="n">normalization_distance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">)</span> </span><span id="distributive_law-142"><a href="#distributive_law-142"><span class="linenos">142</span></a> <span class="n">distance</span> <span class="o">=</span> <span class="n">normalization_distance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="n">dnf</span><span class="p">)</span>
</span><span id="distributive_law-143"><a href="#distributive_law-143"><span class="linenos">143</span></a>
</span><span id="distributive_law-144"><a href="#distributive_law-144"><span class="linenos">144</span></a> <span class="k">if</span> <span class="n">distance</span> <span class="o">&gt;</span> <span class="n">max_distance</span><span class="p">:</span>
</span><span id="distributive_law-145"><a href="#distributive_law-145"><span class="linenos">145</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;Normalization distance </span><span class="si">{</span><span class="n">distance</span><span class="si">}</span><span class="s2"> exceeds max </span><span class="si">{</span><span class="n">max_distance</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">)</span>
</span><span id="distributive_law-146"><a href="#distributive_law-146"><span class="linenos">146</span></a> </span><span id="distributive_law-146"><a href="#distributive_law-146"><span class="linenos">146</span></a>
</span><span id="distributive_law-147"><a href="#distributive_law-147"><span class="linenos">147</span></a> <span class="k">if</span> <span class="n">distance</span> <span class="o">&gt;</span> <span class="n">max_distance</span><span class="p">:</span> </span><span id="distributive_law-147"><a href="#distributive_law-147"><span class="linenos">147</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">e</span><span class="p">:</span> <span class="n">distributive_law</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">dnf</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">))</span>
</span><span id="distributive_law-148"><a href="#distributive_law-148"><span class="linenos">148</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">&quot;Normalization distance </span><span class="si">{</span><span class="n">distance</span><span class="si">}</span><span class="s2"> exceeds max </span><span class="si">{</span><span class="n">max_distance</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">)</span> </span><span id="distributive_law-148"><a href="#distributive_law-148"><span class="linenos">148</span></a> <span class="n">to_exp</span><span class="p">,</span> <span class="n">from_exp</span> <span class="o">=</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">)</span> <span class="k">if</span> <span class="n">dnf</span> <span class="k">else</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span>
</span><span id="distributive_law-149"><a href="#distributive_law-149"><span class="linenos">149</span></a> </span><span id="distributive_law-149"><a href="#distributive_law-149"><span class="linenos">149</span></a>
</span><span id="distributive_law-150"><a href="#distributive_law-150"><span class="linenos">150</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">e</span><span class="p">:</span> <span class="n">distributive_law</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="n">dnf</span><span class="p">,</span> <span class="n">max_distance</span><span class="p">,</span> <span class="n">generate</span><span class="p">))</span> </span><span id="distributive_law-150"><a href="#distributive_law-150"><span class="linenos">150</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">from_exp</span><span class="p">):</span>
</span><span id="distributive_law-151"><a href="#distributive_law-151"><span class="linenos">151</span></a> <span class="n">to_exp</span><span class="p">,</span> <span class="n">from_exp</span> <span class="o">=</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">)</span> <span class="k">if</span> <span class="n">dnf</span> <span class="k">else</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">And</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span> </span><span id="distributive_law-151"><a href="#distributive_law-151"><span class="linenos">151</span></a> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">unnest_operands</span><span class="p">()</span>
</span><span id="distributive_law-152"><a href="#distributive_law-152"><span class="linenos">152</span></a> </span><span id="distributive_law-152"><a href="#distributive_law-152"><span class="linenos">152</span></a>
</span><span id="distributive_law-153"><a href="#distributive_law-153"><span class="linenos">153</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">from_exp</span><span class="p">):</span> </span><span id="distributive_law-153"><a href="#distributive_law-153"><span class="linenos">153</span></a> <span class="n">from_func</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span> <span class="k">if</span> <span class="n">from_exp</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span>
</span><span id="distributive_law-154"><a href="#distributive_law-154"><span class="linenos">154</span></a> <span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">unnest_operands</span><span class="p">()</span> </span><span id="distributive_law-154"><a href="#distributive_law-154"><span class="linenos">154</span></a> <span class="n">to_func</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span> <span class="k">if</span> <span class="n">to_exp</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span>
</span><span id="distributive_law-155"><a href="#distributive_law-155"><span class="linenos">155</span></a> </span><span id="distributive_law-155"><a href="#distributive_law-155"><span class="linenos">155</span></a>
</span><span id="distributive_law-156"><a href="#distributive_law-156"><span class="linenos">156</span></a> <span class="n">from_func</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span> <span class="k">if</span> <span class="n">from_exp</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span> </span><span id="distributive_law-156"><a href="#distributive_law-156"><span class="linenos">156</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">):</span>
</span><span id="distributive_law-157"><a href="#distributive_law-157"><span class="linenos">157</span></a> <span class="n">to_func</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span> <span class="k">if</span> <span class="n">to_exp</span> <span class="o">==</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span> </span><span id="distributive_law-157"><a href="#distributive_law-157"><span class="linenos">157</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="nb">tuple</span><span class="p">(</span><span class="n">a</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">)))</span> <span class="o">&gt;</span> <span class="nb">len</span><span class="p">(</span><span class="nb">tuple</span><span class="p">(</span><span class="n">b</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">))):</span>
</span><span id="distributive_law-158"><a href="#distributive_law-158"><span class="linenos">158</span></a> </span><span id="distributive_law-158"><a href="#distributive_law-158"><span class="linenos">158</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">)</span>
</span><span id="distributive_law-159"><a href="#distributive_law-159"><span class="linenos">159</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">):</span> </span><span id="distributive_law-159"><a href="#distributive_law-159"><span class="linenos">159</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">)</span>
</span><span id="distributive_law-160"><a href="#distributive_law-160"><span class="linenos">160</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="nb">tuple</span><span class="p">(</span><span class="n">a</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">)))</span> <span class="o">&gt;</span> <span class="nb">len</span><span class="p">(</span><span class="nb">tuple</span><span class="p">(</span><span class="n">b</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">))):</span> </span><span id="distributive_law-160"><a href="#distributive_law-160"><span class="linenos">160</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">):</span>
</span><span id="distributive_law-161"><a href="#distributive_law-161"><span class="linenos">161</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">,</span> <span class="n">generate</span><span class="p">)</span> </span><span id="distributive_law-161"><a href="#distributive_law-161"><span class="linenos">161</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">)</span>
</span><span id="distributive_law-162"><a href="#distributive_law-162"><span class="linenos">162</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">,</span> <span class="n">generate</span><span class="p">)</span> </span><span id="distributive_law-162"><a href="#distributive_law-162"><span class="linenos">162</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">):</span>
</span><span id="distributive_law-163"><a href="#distributive_law-163"><span class="linenos">163</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">):</span> </span><span id="distributive_law-163"><a href="#distributive_law-163"><span class="linenos">163</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">)</span>
</span><span id="distributive_law-164"><a href="#distributive_law-164"><span class="linenos">164</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">,</span> <span class="n">generate</span><span class="p">)</span> </span><span id="distributive_law-164"><a href="#distributive_law-164"><span class="linenos">164</span></a>
</span><span id="distributive_law-165"><a href="#distributive_law-165"><span class="linenos">165</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">to_exp</span><span class="p">):</span> </span><span id="distributive_law-165"><a href="#distributive_law-165"><span class="linenos">165</span></a> <span class="k">return</span> <span class="n">expression</span>
</span><span id="distributive_law-166"><a href="#distributive_law-166"><span class="linenos">166</span></a> <span class="k">return</span> <span class="n">_distribute</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">from_func</span><span class="p">,</span> <span class="n">to_func</span><span class="p">,</span> <span class="n">generate</span><span class="p">)</span>
</span><span id="distributive_law-167"><a href="#distributive_law-167"><span class="linenos">167</span></a>
</span><span id="distributive_law-168"><a href="#distributive_law-168"><span class="linenos">168</span></a> <span class="k">return</span> <span class="n">expression</span>
</span></pre></div> </span></pre></div>

52
docs/sqlglot/optimizer/normalize_identifiers.html
View file

@ -60,7 +60,7 @@
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a> </span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a>
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">import</span> <span class="nn">typing</span> <span class="k">as</span> <span class="nn">t</span> </span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">import</span> <span class="nn">typing</span> <span class="k">as</span> <span class="nn">t</span>
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a> </span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a>
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">exp</span><span class="p">,</span> <span class="n">parse_one</span> </span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">ParseError</span><span class="p">,</span> <span class="n">exp</span><span class="p">,</span> <span class="n">parse_one</span>
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot._typing</span> <span class="kn">import</span> <span class="n">E</span> </span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot._typing</span> <span class="kn">import</span> <span class="n">E</span>
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">Dialect</span><span class="p">,</span> <span class="n">DialectType</span> </span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">Dialect</span><span class="p">,</span> <span class="n">DialectType</span>
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a> </span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a>
@ -107,17 +107,20 @@
</span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a><span class="sd"> The transformed expression.</span> </span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a><span class="sd"> The transformed expression.</span>
</span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span> </span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
</span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">parse_one</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span> </span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a> <span class="k">try</span><span class="p">:</span>
</span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a> </span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">parse_one</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span>
</span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span> </span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a> <span class="k">except</span> <span class="n">ParseError</span><span class="p">:</span>
</span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a> </span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a> <span class="k">def</span> <span class="nf">_normalize</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">E</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span> </span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a>
</span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">):</span> </span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">_normalize</span><span class="p">)</span> </span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a>
</span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span> </span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a> <span class="k">def</span> <span class="nf">_normalize</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">E</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span>
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="k">return</span> <span class="n">node</span> </span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">):</span>
</span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a> </span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">_normalize</span><span class="p">)</span>
</span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a> <span class="k">return</span> <span class="n">_normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="L-63"><a href="#L-63"><span class="linenos">63</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="L-64"><a href="#L-64"><span class="linenos">64</span></a>
</span><span id="L-65"><a href="#L-65"><span class="linenos">65</span></a> <span class="k">return</span> <span class="n">_normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span></pre></div> </span></pre></div>
@ -165,17 +168,20 @@
</span><span id="normalize_identifiers-50"><a href="#normalize_identifiers-50"><span class="linenos">50</span></a><span class="sd"> The transformed expression.</span> </span><span id="normalize_identifiers-50"><a href="#normalize_identifiers-50"><span class="linenos">50</span></a><span class="sd"> The transformed expression.</span>
</span><span id="normalize_identifiers-51"><a href="#normalize_identifiers-51"><span class="linenos">51</span></a><span class="sd"> &quot;&quot;&quot;</span> </span><span id="normalize_identifiers-51"><a href="#normalize_identifiers-51"><span class="linenos">51</span></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="normalize_identifiers-52"><a href="#normalize_identifiers-52"><span class="linenos">52</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span> </span><span id="normalize_identifiers-52"><a href="#normalize_identifiers-52"><span class="linenos">52</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
</span><span id="normalize_identifiers-53"><a href="#normalize_identifiers-53"><span class="linenos">53</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">parse_one</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span> </span><span id="normalize_identifiers-53"><a href="#normalize_identifiers-53"><span class="linenos">53</span></a> <span class="k">try</span><span class="p">:</span>
</span><span id="normalize_identifiers-54"><a href="#normalize_identifiers-54"><span class="linenos">54</span></a> </span><span id="normalize_identifiers-54"><a href="#normalize_identifiers-54"><span class="linenos">54</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">parse_one</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span>
</span><span id="normalize_identifiers-55"><a href="#normalize_identifiers-55"><span class="linenos">55</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span> </span><span id="normalize_identifiers-55"><a href="#normalize_identifiers-55"><span class="linenos">55</span></a> <span class="k">except</span> <span class="n">ParseError</span><span class="p">:</span>
</span><span id="normalize_identifiers-56"><a href="#normalize_identifiers-56"><span class="linenos">56</span></a> </span><span id="normalize_identifiers-56"><a href="#normalize_identifiers-56"><span class="linenos">56</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span><span id="normalize_identifiers-57"><a href="#normalize_identifiers-57"><span class="linenos">57</span></a> <span class="k">def</span> <span class="nf">_normalize</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">E</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span> </span><span id="normalize_identifiers-57"><a href="#normalize_identifiers-57"><span class="linenos">57</span></a>
</span><span id="normalize_identifiers-58"><a href="#normalize_identifiers-58"><span class="linenos">58</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">):</span> </span><span id="normalize_identifiers-58"><a href="#normalize_identifiers-58"><span class="linenos">58</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span>
</span><span id="normalize_identifiers-59"><a href="#normalize_identifiers-59"><span class="linenos">59</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">_normalize</span><span class="p">)</span> </span><span id="normalize_identifiers-59"><a href="#normalize_identifiers-59"><span class="linenos">59</span></a>
</span><span id="normalize_identifiers-60"><a href="#normalize_identifiers-60"><span class="linenos">60</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span> </span><span id="normalize_identifiers-60"><a href="#normalize_identifiers-60"><span class="linenos">60</span></a> <span class="k">def</span> <span class="nf">_normalize</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">E</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">E</span><span class="p">:</span>
</span><span id="normalize_identifiers-61"><a href="#normalize_identifiers-61"><span class="linenos">61</span></a> <span class="k">return</span> <span class="n">node</span> </span><span id="normalize_identifiers-61"><a href="#normalize_identifiers-61"><span class="linenos">61</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">meta</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;case_sensitive&quot;</span><span class="p">):</span>
</span><span id="normalize_identifiers-62"><a href="#normalize_identifiers-62"><span class="linenos">62</span></a> </span><span id="normalize_identifiers-62"><a href="#normalize_identifiers-62"><span class="linenos">62</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">replace_children</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">_normalize</span><span class="p">)</span>
</span><span id="normalize_identifiers-63"><a href="#normalize_identifiers-63"><span class="linenos">63</span></a> <span class="k">return</span> <span class="n">_normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span> </span><span id="normalize_identifiers-63"><a href="#normalize_identifiers-63"><span class="linenos">63</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">dialect</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
</span><span id="normalize_identifiers-64"><a href="#normalize_identifiers-64"><span class="linenos">64</span></a> <span class="k">return</span> <span class="n">node</span>
</span><span id="normalize_identifiers-65"><a href="#normalize_identifiers-65"><span class="linenos">65</span></a>
</span><span id="normalize_identifiers-66"><a href="#normalize_identifiers-66"><span class="linenos">66</span></a> <span class="k">return</span> <span class="n">_normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
</span></pre></div> </span></pre></div>

2
docs/sqlglot/optimizer/optimizer.html
View file

File diff suppressed because one or more lines are too long

3337
docs/sqlglot/optimizer/simplify.html
View file

File diff suppressed because it is too large Load diff

19898
docs/sqlglot/parser.html
View file

File diff suppressed because one or more lines are too long

2
docs/sqlglot/time.html
View file

File diff suppressed because one or more lines are too long

4
docs/sqlglot/tokens.html
View file

@ -7875,7 +7875,7 @@
<div class="attr variable"> <div class="attr variable">
<span class="name">COMMANDS</span> = <span class="name">COMMANDS</span> =
<input id="Tokenizer.COMMANDS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1"> <input id="Tokenizer.COMMANDS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
<label class="view-value-button pdoc-button" for="Tokenizer.COMMANDS-view-value"></label><span class="default_value">{&lt;<a href="#TokenType.FETCH">TokenType.FETCH</a>: &#39;FETCH&#39;&gt;, &lt;<a href="#TokenType.SHOW">TokenType.SHOW</a>: &#39;SHOW&#39;&gt;, &lt;<a href="#TokenType.EXECUTE">TokenType.EXECUTE</a>: &#39;EXECUTE&#39;&gt;, &lt;<a href="#TokenType.COMMAND">TokenType.COMMAND</a>: &#39;COMMAND&#39;&gt;}</span> <label class="view-value-button pdoc-button" for="Tokenizer.COMMANDS-view-value"></label><span class="default_value">{&lt;<a href="#TokenType.FETCH">TokenType.FETCH</a>: &#39;FETCH&#39;&gt;, &lt;<a href="#TokenType.COMMAND">TokenType.COMMAND</a>: &#39;COMMAND&#39;&gt;, &lt;<a href="#TokenType.SHOW">TokenType.SHOW</a>: &#39;SHOW&#39;&gt;, &lt;<a href="#TokenType.EXECUTE">TokenType.EXECUTE</a>: &#39;EXECUTE&#39;&gt;}</span>
</div> </div>
@ -7887,7 +7887,7 @@
<div id="Tokenizer.COMMAND_PREFIX_TOKENS" class="classattr"> <div id="Tokenizer.COMMAND_PREFIX_TOKENS" class="classattr">
<div class="attr variable"> <div class="attr variable">
<span class="name">COMMAND_PREFIX_TOKENS</span> = <span class="name">COMMAND_PREFIX_TOKENS</span> =
<span class="default_value">{&lt;<a href="#TokenType.BEGIN">TokenType.BEGIN</a>: &#39;BEGIN&#39;&gt;, &lt;<a href="#TokenType.SEMICOLON">TokenType.SEMICOLON</a>: &#39;SEMICOLON&#39;&gt;}</span> <span class="default_value">{&lt;<a href="#TokenType.SEMICOLON">TokenType.SEMICOLON</a>: &#39;SEMICOLON&#39;&gt;, &lt;<a href="#TokenType.BEGIN">TokenType.BEGIN</a>: &#39;BEGIN&#39;&gt;}</span>
</div> </div>

1259
docs/sqlglot/transforms.html
View file

File diff suppressed because it is too large Load diff

2
sqlglot/__init__.py
View file

@ -158,6 +158,6 @@ def transpile(
""" """
write = (read if write is None else write) if identity else write write = (read if write is None else write) if identity else write
return [ return [
Dialect.get_or_raise(write)().generate(expression, **opts) Dialect.get_or_raise(write)().generate(expression, copy=False, **opts) if expression else ""
for expression in parse(sql, read, error_level=error_level) for expression in parse(sql, read, error_level=error_level)
] ]

2
sqlglot/dialects/bigquery.py
View file

@ -69,7 +69,6 @@ def _create_sql(self: BigQuery.Generator, expression: exp.Create) -> str:
returns = expression.find(exp.ReturnsProperty) returns = expression.find(exp.ReturnsProperty)
if kind.upper() == "FUNCTION" and returns and returns.args.get("is_table"): if kind.upper() == "FUNCTION" and returns and returns.args.get("is_table"):
expression = expression.copy()
expression.set("kind", "TABLE FUNCTION") expression.set("kind", "TABLE FUNCTION")
if isinstance(expression.expression, (exp.Subquery, exp.Literal)): if isinstance(expression.expression, (exp.Subquery, exp.Literal)):
@ -699,6 +698,5 @@ class BigQuery(Dialect):
def version_sql(self, expression: exp.Version) -> str: def version_sql(self, expression: exp.Version) -> str:
if expression.name == "TIMESTAMP": if expression.name == "TIMESTAMP":
expression = expression.copy()
expression.set("this", "SYSTEM_TIME") expression.set("this", "SYSTEM_TIME")
return super().version_sql(expression) return super().version_sql(expression)

1
sqlglot/dialects/clickhouse.py
View file

@ -461,7 +461,6 @@ class ClickHouse(Dialect):
def safeconcat_sql(self, expression: exp.SafeConcat) -> str: def safeconcat_sql(self, expression: exp.SafeConcat) -> str:
# Clickhouse errors out if we try to cast a NULL value to TEXT # Clickhouse errors out if we try to cast a NULL value to TEXT
expression = expression.copy()
return self.func( return self.func(
"CONCAT", "CONCAT",
*[ *[

9
sqlglot/dialects/databricks.py
View file

@ -35,7 +35,7 @@ class Databricks(Spark):
exp.DatetimeSub: lambda self, e: self.func( exp.DatetimeSub: lambda self, e: self.func(
"TIMESTAMPADD", "TIMESTAMPADD",
e.text("unit"), e.text("unit"),
exp.Mul(this=e.expression.copy(), expression=exp.Literal.number(-1)), exp.Mul(this=e.expression, expression=exp.Literal.number(-1)),
e.this, e.this,
), ),
exp.DatetimeDiff: lambda self, e: self.func( exp.DatetimeDiff: lambda self, e: self.func(
@ -63,21 +63,14 @@ class Databricks(Spark):
and kind.this in exp.DataType.INTEGER_TYPES and kind.this in exp.DataType.INTEGER_TYPES
): ):
# only BIGINT generated identity constraints are supported # only BIGINT generated identity constraints are supported
expression = expression.copy()
expression.set("kind", exp.DataType.build("bigint")) expression.set("kind", exp.DataType.build("bigint"))
return super().columndef_sql(expression, sep) return super().columndef_sql(expression, sep)
def generatedasidentitycolumnconstraint_sql( def generatedasidentitycolumnconstraint_sql(
self, expression: exp.GeneratedAsIdentityColumnConstraint self, expression: exp.GeneratedAsIdentityColumnConstraint
) -> str: ) -> str:
expression = expression.copy()
expression.set("this", True) # trigger ALWAYS in super class expression.set("this", True) # trigger ALWAYS in super class
return super().generatedasidentitycolumnconstraint_sql(expression) return super().generatedasidentitycolumnconstraint_sql(expression)
class Tokenizer(Spark.Tokenizer): class Tokenizer(Spark.Tokenizer):
HEX_STRINGS = [] HEX_STRINGS = []
SINGLE_TOKENS = {
**Spark.Tokenizer.SINGLE_TOKENS,
"$": TokenType.PARAMETER,
}

29
sqlglot/dialects/dialect.py
View file

@ -315,11 +315,14 @@ class Dialect(metaclass=_Dialect):
) -> t.List[t.Optional[exp.Expression]]: ) -> t.List[t.Optional[exp.Expression]]:
return self.parser(**opts).parse_into(expression_type, self.tokenize(sql), sql) return self.parser(**opts).parse_into(expression_type, self.tokenize(sql), sql)
def generate(self, expression: t.Optional[exp.Expression], **opts) -> str: def generate(self, expression: exp.Expression, copy: bool = True, **opts) -> str:
return self.generator(**opts).generate(expression) return self.generator(**opts).generate(expression, copy=copy)
def transpile(self, sql: str, **opts) -> t.List[str]: def transpile(self, sql: str, **opts) -> t.List[str]:
return [self.generate(expression, **opts) for expression in self.parse(sql)] return [
self.generate(expression, copy=False, **opts) if expression else ""
for expression in self.parse(sql)
]
def tokenize(self, sql: str) -> t.List[Token]: def tokenize(self, sql: str) -> t.List[Token]:
return self.tokenizer.tokenize(sql) return self.tokenizer.tokenize(sql)
@ -380,9 +383,7 @@ def inline_array_sql(self: Generator, expression: exp.Array) -> str:
def no_ilike_sql(self: Generator, expression: exp.ILike) -> str: def no_ilike_sql(self: Generator, expression: exp.ILike) -> str:
return self.like_sql( return self.like_sql(
exp.Like( exp.Like(this=exp.Lower(this=expression.this), expression=expression.expression)
this=exp.Lower(this=expression.this.copy()), expression=expression.expression.copy()
)
) )
@ -518,7 +519,6 @@ def create_with_partitions_sql(self: Generator, expression: exp.Create) -> str:
is_partitionable = expression.args.get("kind") in ("TABLE", "VIEW") is_partitionable = expression.args.get("kind") in ("TABLE", "VIEW")
if has_schema and is_partitionable: if has_schema and is_partitionable:
expression = expression.copy()
prop = expression.find(exp.PartitionedByProperty) prop = expression.find(exp.PartitionedByProperty)
if prop and prop.this and not isinstance(prop.this, exp.Schema): if prop and prop.this and not isinstance(prop.this, exp.Schema):
schema = expression.this schema = expression.this
@ -583,7 +583,7 @@ def date_add_interval_sql(
this = self.sql(expression, "this") this = self.sql(expression, "this")
unit = expression.args.get("unit") unit = expression.args.get("unit")
unit = exp.var(unit.name.upper() if unit else "DAY") unit = exp.var(unit.name.upper() if unit else "DAY")
interval = exp.Interval(this=expression.expression.copy(), unit=unit) interval = exp.Interval(this=expression.expression, unit=unit)
return f"{data_type}_{kind}({this}, {self.sql(interval)})" return f"{data_type}_{kind}({this}, {self.sql(interval)})"
return func return func
@ -621,7 +621,6 @@ def strposition_to_locate_sql(self: Generator, expression: exp.StrPosition) -> s
def left_to_substring_sql(self: Generator, expression: exp.Left) -> str: def left_to_substring_sql(self: Generator, expression: exp.Left) -> str:
expression = expression.copy()
return self.sql( return self.sql(
exp.Substring( exp.Substring(
this=expression.this, start=exp.Literal.number(1), length=expression.expression this=expression.this, start=exp.Literal.number(1), length=expression.expression
@ -630,7 +629,6 @@ def left_to_substring_sql(self: Generator, expression: exp.Left) -> str:
def right_to_substring_sql(self: Generator, expression: exp.Left) -> str: def right_to_substring_sql(self: Generator, expression: exp.Left) -> str:
expression = expression.copy()
return self.sql( return self.sql(
exp.Substring( exp.Substring(
this=expression.this, this=expression.this,
@ -675,7 +673,7 @@ def count_if_to_sum(self: Generator, expression: exp.CountIf) -> str:
cond = expression.this.expressions[0] cond = expression.this.expressions[0]
self.unsupported("DISTINCT is not supported when converting COUNT_IF to SUM") self.unsupported("DISTINCT is not supported when converting COUNT_IF to SUM")
return self.func("sum", exp.func("if", cond.copy(), 1, 0)) return self.func("sum", exp.func("if", cond, 1, 0))
def trim_sql(self: Generator, expression: exp.Trim) -> str: def trim_sql(self: Generator, expression: exp.Trim) -> str:
@ -716,12 +714,10 @@ def ts_or_ds_to_date_sql(dialect: str) -> t.Callable:
def concat_to_dpipe_sql(self: Generator, expression: exp.Concat | exp.SafeConcat) -> str: def concat_to_dpipe_sql(self: Generator, expression: exp.Concat | exp.SafeConcat) -> str:
expression = expression.copy()
return self.sql(reduce(lambda x, y: exp.DPipe(this=x, expression=y), expression.expressions)) return self.sql(reduce(lambda x, y: exp.DPipe(this=x, expression=y), expression.expressions))
def concat_ws_to_dpipe_sql(self: Generator, expression: exp.ConcatWs) -> str: def concat_ws_to_dpipe_sql(self: Generator, expression: exp.ConcatWs) -> str:
expression = expression.copy()
delim, *rest_args = expression.expressions delim, *rest_args = expression.expressions
return self.sql( return self.sql(
reduce( reduce(
@ -809,13 +805,6 @@ def isnull_to_is_null(args: t.List) -> exp.Expression:
return exp.Paren(this=exp.Is(this=seq_get(args, 0), expression=exp.null())) return exp.Paren(this=exp.Is(this=seq_get(args, 0), expression=exp.null()))
def move_insert_cte_sql(self: Generator, expression: exp.Insert) -> str:
if expression.expression.args.get("with"):
expression = expression.copy()
expression.set("with", expression.expression.args["with"].pop())
return self.insert_sql(expression)
def generatedasidentitycolumnconstraint_sql( def generatedasidentitycolumnconstraint_sql(
self: Generator, expression: exp.GeneratedAsIdentityColumnConstraint self: Generator, expression: exp.GeneratedAsIdentityColumnConstraint
) -> str: ) -> str:

6
sqlglot/dialects/drill.py
View file

@ -20,7 +20,9 @@ def _date_add_sql(kind: str) -> t.Callable[[Drill.Generator, exp.DateAdd | exp.D
def func(self: Drill.Generator, expression: exp.DateAdd | exp.DateSub) -> str: def func(self: Drill.Generator, expression: exp.DateAdd | exp.DateSub) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
unit = exp.var(expression.text("unit").upper() or "DAY") unit = exp.var(expression.text("unit").upper() or "DAY")
return f"DATE_{kind}({this}, {self.sql(exp.Interval(this=expression.expression.copy(), unit=unit))})" return (
f"DATE_{kind}({this}, {self.sql(exp.Interval(this=expression.expression, unit=unit))})"
)
return func return func
@ -147,7 +149,7 @@ class Drill(Dialect):
exp.TimeToUnix: rename_func("UNIX_TIMESTAMP"), exp.TimeToUnix: rename_func("UNIX_TIMESTAMP"),
exp.ToChar: lambda self, e: self.function_fallback_sql(e), exp.ToChar: lambda self, e: self.function_fallback_sql(e),
exp.TryCast: no_trycast_sql, exp.TryCast: no_trycast_sql,
exp.TsOrDsAdd: lambda self, e: f"DATE_ADD(CAST({self.sql(e, 'this')} AS DATE), {self.sql(exp.Interval(this=e.expression.copy(), unit=exp.var('DAY')))})", exp.TsOrDsAdd: lambda self, e: f"DATE_ADD(CAST({self.sql(e, 'this')} AS DATE), {self.sql(exp.Interval(this=e.expression, unit=exp.var('DAY')))})",
exp.TsOrDsToDate: ts_or_ds_to_date_sql("drill"), exp.TsOrDsToDate: ts_or_ds_to_date_sql("drill"),
exp.TsOrDiToDi: lambda self, e: f"CAST(SUBSTR(REPLACE(CAST({self.sql(e, 'this')} AS VARCHAR), '-', ''), 1, 8) AS INT)", exp.TsOrDiToDi: lambda self, e: f"CAST(SUBSTR(REPLACE(CAST({self.sql(e, 'this')} AS VARCHAR), '-', ''), 1, 8) AS INT)",
} }

6
sqlglot/dialects/duckdb.py
View file

@ -36,14 +36,14 @@ from sqlglot.tokens import TokenType
def _ts_or_ds_add_sql(self: DuckDB.Generator, expression: exp.TsOrDsAdd) -> str: def _ts_or_ds_add_sql(self: DuckDB.Generator, expression: exp.TsOrDsAdd) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
unit = self.sql(expression, "unit").strip("'") or "DAY" unit = self.sql(expression, "unit").strip("'") or "DAY"
return f"CAST({this} AS DATE) + {self.sql(exp.Interval(this=expression.expression.copy(), unit=unit))}" return f"CAST({this} AS DATE) + {self.sql(exp.Interval(this=expression.expression, unit=unit))}"
def _date_delta_sql(self: DuckDB.Generator, expression: exp.DateAdd | exp.DateSub) -> str: def _date_delta_sql(self: DuckDB.Generator, expression: exp.DateAdd | exp.DateSub) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
unit = self.sql(expression, "unit").strip("'") or "DAY" unit = self.sql(expression, "unit").strip("'") or "DAY"
op = "+" if isinstance(expression, exp.DateAdd) else "-" op = "+" if isinstance(expression, exp.DateAdd) else "-"
return f"{this} {op} {self.sql(exp.Interval(this=expression.expression.copy(), unit=unit))}" return f"{this} {op} {self.sql(exp.Interval(this=expression.expression, unit=unit))}"
# BigQuery -> DuckDB conversion for the DATE function # BigQuery -> DuckDB conversion for the DATE function
@ -365,7 +365,7 @@ class DuckDB(Dialect):
multiplier = 90 multiplier = 90
if multiplier: if multiplier:
return f"({multiplier} * {super().interval_sql(exp.Interval(this=expression.this.copy(), unit=exp.var('day')))})" return f"({multiplier} * {super().interval_sql(exp.Interval(this=expression.this, unit=exp.var('day')))})"
return super().interval_sql(expression) return super().interval_sql(expression)

17
sqlglot/dialects/hive.py
View file

@ -53,8 +53,6 @@ DIFF_MONTH_SWITCH = ("YEAR", "QUARTER", "MONTH")
def _create_sql(self, expression: exp.Create) -> str: def _create_sql(self, expression: exp.Create) -> str:
expression = expression.copy()
# remove UNIQUE column constraints # remove UNIQUE column constraints
for constraint in expression.find_all(exp.UniqueColumnConstraint): for constraint in expression.find_all(exp.UniqueColumnConstraint):
if constraint.parent: if constraint.parent:
@ -88,7 +86,7 @@ def _add_date_sql(self: Hive.Generator, expression: exp.DateAdd | exp.DateSub) -
if expression.expression.is_number: if expression.expression.is_number:
modified_increment = exp.Literal.number(int(expression.text("expression")) * multiplier) modified_increment = exp.Literal.number(int(expression.text("expression")) * multiplier)
else: else:
modified_increment = expression.expression.copy() modified_increment = expression.expression
if multiplier != 1: if multiplier != 1:
modified_increment = exp.Mul( # type: ignore modified_increment = exp.Mul( # type: ignore
this=modified_increment, expression=exp.Literal.number(multiplier) this=modified_increment, expression=exp.Literal.number(multiplier)
@ -229,6 +227,11 @@ class Hive(Dialect):
STRING_ESCAPES = ["\\"] STRING_ESCAPES = ["\\"]
ENCODE = "utf-8" ENCODE = "utf-8"
SINGLE_TOKENS = {
**tokens.Tokenizer.SINGLE_TOKENS,
"$": TokenType.PARAMETER,
}
KEYWORDS = { KEYWORDS = {
**tokens.Tokenizer.KEYWORDS, **tokens.Tokenizer.KEYWORDS,
"ADD ARCHIVE": TokenType.COMMAND, "ADD ARCHIVE": TokenType.COMMAND,
@ -408,6 +411,7 @@ class Hive(Dialect):
INDEX_ON = "ON TABLE" INDEX_ON = "ON TABLE"
EXTRACT_ALLOWS_QUOTES = False EXTRACT_ALLOWS_QUOTES = False
NVL2_SUPPORTED = False NVL2_SUPPORTED = False
SUPPORTS_NESTED_CTES = False
TYPE_MAPPING = { TYPE_MAPPING = {
**generator.Generator.TYPE_MAPPING, **generator.Generator.TYPE_MAPPING,
@ -521,7 +525,10 @@ class Hive(Dialect):
def parameter_sql(self, expression: exp.Parameter) -> str: def parameter_sql(self, expression: exp.Parameter) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
expression_sql = self.sql(expression, "expression")
parent = expression.parent parent = expression.parent
this = f"{this}:{expression_sql}" if expression_sql else this
if isinstance(parent, exp.EQ) and isinstance(parent.parent, exp.SetItem): if isinstance(parent, exp.EQ) and isinstance(parent.parent, exp.SetItem):
# We need to produce SET key = value instead of SET ${key} = value # We need to produce SET key = value instead of SET ${key} = value
@ -530,8 +537,6 @@ class Hive(Dialect):
return f"${{{this}}}" return f"${{{this}}}"
def schema_sql(self, expression: exp.Schema) -> str: def schema_sql(self, expression: exp.Schema) -> str:
expression = expression.copy()
for ordered in expression.find_all(exp.Ordered): for ordered in expression.find_all(exp.Ordered):
if ordered.args.get("desc") is False: if ordered.args.get("desc") is False:
ordered.set("desc", None) ordered.set("desc", None)
@ -539,8 +544,6 @@ class Hive(Dialect):
return super().schema_sql(expression) return super().schema_sql(expression)
def constraint_sql(self, expression: exp.Constraint) -> str: def constraint_sql(self, expression: exp.Constraint) -> str:
expression = expression.copy()
for prop in list(expression.find_all(exp.Properties)): for prop in list(expression.find_all(exp.Properties)):
prop.pop() prop.pop()

38
sqlglot/dialects/mysql.py
View file

@ -60,9 +60,33 @@ def _date_trunc_sql(self: MySQL.Generator, expression: exp.DateTrunc) -> str:
return f"STR_TO_DATE({concat}, '{date_format}')" return f"STR_TO_DATE({concat}, '{date_format}')"
def _str_to_date(args: t.List) -> exp.StrToDate: # All specifiers for time parts (as opposed to date parts)
date_format = MySQL.format_time(seq_get(args, 1)) # https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#function_date-format
return exp.StrToDate(this=seq_get(args, 0), format=date_format) TIME_SPECIFIERS = {"f", "H", "h", "I", "i", "k", "l", "p", "r", "S", "s", "T"}
def _has_time_specifier(date_format: str) -> bool:
i = 0
length = len(date_format)
while i < length:
if date_format[i] == "%":
i += 1
if i < length and date_format[i] in TIME_SPECIFIERS:
return True
i += 1
return False
def _str_to_date(args: t.List) -> exp.StrToDate | exp.StrToTime:
mysql_date_format = seq_get(args, 1)
date_format = MySQL.format_time(mysql_date_format)
this = seq_get(args, 0)
if mysql_date_format and _has_time_specifier(mysql_date_format.name):
return exp.StrToTime(this=this, format=date_format)
return exp.StrToDate(this=this, format=date_format)
def _str_to_date_sql( def _str_to_date_sql(
@ -93,7 +117,9 @@ def _date_add_sql(
def func(self: MySQL.Generator, expression: exp.Expression) -> str: def func(self: MySQL.Generator, expression: exp.Expression) -> str:
this = self.sql(expression, "this") this = self.sql(expression, "this")
unit = expression.text("unit").upper() or "DAY" unit = expression.text("unit").upper() or "DAY"
return f"DATE_{kind}({this}, {self.sql(exp.Interval(this=expression.expression.copy(), unit=unit))})" return (
f"DATE_{kind}({this}, {self.sql(exp.Interval(this=expression.expression, unit=unit))})"
)
return func return func
@ -110,8 +136,6 @@ def _remove_ts_or_ds_to_date(
args: t.Tuple[str, ...] = ("this",), args: t.Tuple[str, ...] = ("this",),
) -> t.Callable[[MySQL.Generator, exp.Func], str]: ) -> t.Callable[[MySQL.Generator, exp.Func], str]:
def func(self: MySQL.Generator, expression: exp.Func) -> str: def func(self: MySQL.Generator, expression: exp.Func) -> str:
expression = expression.copy()
for arg_key in args: for arg_key in args:
arg = expression.args.get(arg_key) arg = expression.args.get(arg_key)
if isinstance(arg, exp.TsOrDsToDate) and not arg.args.get("format"): if isinstance(arg, exp.TsOrDsToDate) and not arg.args.get("format"):
@ -629,6 +653,7 @@ class MySQL(Dialect):
transforms.eliminate_distinct_on, transforms.eliminate_distinct_on,
transforms.eliminate_semi_and_anti_joins, transforms.eliminate_semi_and_anti_joins,
transforms.eliminate_qualify, transforms.eliminate_qualify,
transforms.eliminate_full_outer_join,
] ]
), ),
exp.StrPosition: strposition_to_locate_sql, exp.StrPosition: strposition_to_locate_sql,
@ -728,7 +753,6 @@ class MySQL(Dialect):
to = self.CAST_MAPPING.get(expression.to.this) to = self.CAST_MAPPING.get(expression.to.this)
if to: if to:
expression = expression.copy()
expression.to.set("this", to) expression.to.set("this", to)
return super().cast_sql(expression) return super().cast_sql(expression)

12
sqlglot/dialects/postgres.py
View file

@ -43,8 +43,6 @@ DATE_DIFF_FACTOR = {
def _date_add_sql(kind: str) -> t.Callable[[Postgres.Generator, exp.DateAdd | exp.DateSub], str]: def _date_add_sql(kind: str) -> t.Callable[[Postgres.Generator, exp.DateAdd | exp.DateSub], str]:
def func(self: Postgres.Generator, expression: exp.DateAdd | exp.DateSub) -> str: def func(self: Postgres.Generator, expression: exp.DateAdd | exp.DateSub) -> str:
expression = expression.copy()
this = self.sql(expression, "this") this = self.sql(expression, "this")
unit = expression.args.get("unit") unit = expression.args.get("unit")
@ -96,7 +94,6 @@ def _substring_sql(self: Postgres.Generator, expression: exp.Substring) -> str:
def _string_agg_sql(self: Postgres.Generator, expression: exp.GroupConcat) -> str: def _string_agg_sql(self: Postgres.Generator, expression: exp.GroupConcat) -> str:
expression = expression.copy()
separator = expression.args.get("separator") or exp.Literal.string(",") separator = expression.args.get("separator") or exp.Literal.string(",")
order = "" order = ""
@ -119,7 +116,6 @@ def _auto_increment_to_serial(expression: exp.Expression) -> exp.Expression:
auto = expression.find(exp.AutoIncrementColumnConstraint) auto = expression.find(exp.AutoIncrementColumnConstraint)
if auto: if auto:
expression = expression.copy()
expression.args["constraints"].remove(auto.parent) expression.args["constraints"].remove(auto.parent)
kind = expression.args["kind"] kind = expression.args["kind"]
@ -134,7 +130,9 @@ def _auto_increment_to_serial(expression: exp.Expression) -> exp.Expression:
def _serial_to_generated(expression: exp.Expression) -> exp.Expression: def _serial_to_generated(expression: exp.Expression) -> exp.Expression:
kind = expression.args["kind"] kind = expression.args.get("kind")
if not kind:
return expression
if kind.this == exp.DataType.Type.SERIAL: if kind.this == exp.DataType.Type.SERIAL:
data_type = exp.DataType(this=exp.DataType.Type.INT) data_type = exp.DataType(this=exp.DataType.Type.INT)
@ -146,7 +144,6 @@ def _serial_to_generated(expression: exp.Expression) -> exp.Expression:
data_type = None data_type = None
if data_type: if data_type:
expression = expression.copy()
expression.args["kind"].replace(data_type) expression.args["kind"].replace(data_type)
constraints = expression.args["constraints"] constraints = expression.args["constraints"]
generated = exp.ColumnConstraint(kind=exp.GeneratedAsIdentityColumnConstraint(this=False)) generated = exp.ColumnConstraint(kind=exp.GeneratedAsIdentityColumnConstraint(this=False))
@ -409,6 +406,7 @@ class Postgres(Dialect):
exp.MapFromEntries: no_map_from_entries_sql, exp.MapFromEntries: no_map_from_entries_sql,
exp.Min: min_or_least, exp.Min: min_or_least,
exp.Merge: transforms.preprocess([_remove_target_from_merge]), exp.Merge: transforms.preprocess([_remove_target_from_merge]),
exp.PartitionedByProperty: lambda self, e: f"PARTITION BY {self.sql(e, 'this')}",
exp.PercentileCont: transforms.preprocess( exp.PercentileCont: transforms.preprocess(
[transforms.add_within_group_for_percentiles] [transforms.add_within_group_for_percentiles]
), ),
@ -445,6 +443,7 @@ class Postgres(Dialect):
PROPERTIES_LOCATION = { PROPERTIES_LOCATION = {
**generator.Generator.PROPERTIES_LOCATION, **generator.Generator.PROPERTIES_LOCATION,
exp.PartitionedByProperty: exp.Properties.Location.POST_SCHEMA,
exp.TransientProperty: exp.Properties.Location.UNSUPPORTED, exp.TransientProperty: exp.Properties.Location.UNSUPPORTED,
exp.VolatileProperty: exp.Properties.Location.UNSUPPORTED, exp.VolatileProperty: exp.Properties.Location.UNSUPPORTED,
} }
@ -452,7 +451,6 @@ class Postgres(Dialect):
def bracket_sql(self, expression: exp.Bracket) -> str: def bracket_sql(self, expression: exp.Bracket) -> str:
"""Forms like ARRAY[1, 2, 3][3] aren't allowed; we need to wrap the ARRAY.""" """Forms like ARRAY[1, 2, 3][3] aren't allowed; we need to wrap the ARRAY."""
if isinstance(expression.this, exp.Array): if isinstance(expression.this, exp.Array):
expression = expression.copy()
expression.set("this", exp.paren(expression.this, copy=False)) expression.set("this", exp.paren(expression.this, copy=False))
return super().bracket_sql(expression) return super().bracket_sql(expression)

9
sqlglot/dialects/presto.py
View file

@ -36,7 +36,6 @@ def _approx_distinct_sql(self: Presto.Generator, expression: exp.ApproxDistinct)
def _explode_to_unnest_sql(self: Presto.Generator, expression: exp.Lateral) -> str: def _explode_to_unnest_sql(self: Presto.Generator, expression: exp.Lateral) -> str:
if isinstance(expression.this, exp.Explode): if isinstance(expression.this, exp.Explode):
expression = expression.copy()
return self.sql( return self.sql(
exp.Join( exp.Join(
this=exp.Unnest( this=exp.Unnest(
@ -72,7 +71,6 @@ def _schema_sql(self: Presto.Generator, expression: exp.Schema) -> str:
for schema in expression.parent.find_all(exp.Schema): for schema in expression.parent.find_all(exp.Schema):
column_defs = schema.find_all(exp.ColumnDef) column_defs = schema.find_all(exp.ColumnDef)
if column_defs and isinstance(schema.parent, exp.Property): if column_defs and isinstance(schema.parent, exp.Property):
expression = expression.copy()
expression.expressions.extend(column_defs) expression.expressions.extend(column_defs)
return self.schema_sql(expression) return self.schema_sql(expression)
@ -407,12 +405,10 @@ class Presto(Dialect):
target_type = None target_type = None
if target_type and target_type.is_type("timestamp"): if target_type and target_type.is_type("timestamp"):
to = target_type.copy()
if target_type is start.to: if target_type is start.to:
end = exp.cast(end, to) end = exp.cast(end, target_type)
else: else:
start = exp.cast(start, to) start = exp.cast(start, target_type)
return self.func("SEQUENCE", start, end, step) return self.func("SEQUENCE", start, end, step)
@ -432,6 +428,5 @@ class Presto(Dialect):
kind = expression.args["kind"] kind = expression.args["kind"]
schema = expression.this schema = expression.this
if kind == "VIEW" and schema.expressions: if kind == "VIEW" and schema.expressions:
expression = expression.copy()
expression.this.set("expressions", None) expression.this.set("expressions", None)
return super().create_sql(expression) return super().create_sql(expression)

18
sqlglot/dialects/redshift.py
View file

@ -27,6 +27,14 @@ def _parse_date_add(args: t.List) -> exp.DateAdd:
) )
def _parse_datediff(args: t.List) -> exp.DateDiff:
return exp.DateDiff(
this=exp.TsOrDsToDate(this=seq_get(args, 2)),
expression=exp.TsOrDsToDate(this=seq_get(args, 1)),
unit=seq_get(args, 0),
)
class Redshift(Postgres): class Redshift(Postgres):
# https://docs.aws.amazon.com/redshift/latest/dg/r_names.html # https://docs.aws.amazon.com/redshift/latest/dg/r_names.html
RESOLVES_IDENTIFIERS_AS_UPPERCASE = None RESOLVES_IDENTIFIERS_AS_UPPERCASE = None
@ -51,11 +59,9 @@ class Redshift(Postgres):
), ),
"DATEADD": _parse_date_add, "DATEADD": _parse_date_add,
"DATE_ADD": _parse_date_add, "DATE_ADD": _parse_date_add,
"DATEDIFF": lambda args: exp.DateDiff( "DATEDIFF": _parse_datediff,
this=exp.TsOrDsToDate(this=seq_get(args, 2)), "DATE_DIFF": _parse_datediff,
expression=exp.TsOrDsToDate(this=seq_get(args, 1)), "LISTAGG": exp.GroupConcat.from_arg_list,
unit=seq_get(args, 0),
),
"STRTOL": exp.FromBase.from_arg_list, "STRTOL": exp.FromBase.from_arg_list,
} }
@ -175,6 +181,7 @@ class Redshift(Postgres):
exp.GeneratedAsIdentityColumnConstraint: generatedasidentitycolumnconstraint_sql, exp.GeneratedAsIdentityColumnConstraint: generatedasidentitycolumnconstraint_sql,
exp.JSONExtract: _json_sql, exp.JSONExtract: _json_sql,
exp.JSONExtractScalar: _json_sql, exp.JSONExtractScalar: _json_sql,
exp.GroupConcat: rename_func("LISTAGG"),
exp.ParseJSON: rename_func("JSON_PARSE"), exp.ParseJSON: rename_func("JSON_PARSE"),
exp.SafeConcat: concat_to_dpipe_sql, exp.SafeConcat: concat_to_dpipe_sql,
exp.Select: transforms.preprocess( exp.Select: transforms.preprocess(
@ -207,7 +214,6 @@ class Redshift(Postgres):
`TEXT` to `VARCHAR`. `TEXT` to `VARCHAR`.
""" """
if expression.is_type("text"): if expression.is_type("text"):
expression = expression.copy()
expression.set("this", exp.DataType.Type.VARCHAR) expression.set("this", exp.DataType.Type.VARCHAR)
precision = expression.args.get("expressions") precision = expression.args.get("expressions")

7
sqlglot/dialects/snowflake.py
View file

@ -32,7 +32,7 @@ def _check_int(s: str) -> bool:
# from https://docs.snowflake.com/en/sql-reference/functions/to_timestamp.html # from https://docs.snowflake.com/en/sql-reference/functions/to_timestamp.html
def _parse_to_timestamp(args: t.List) -> t.Union[exp.StrToTime, exp.UnixToTime]: def _parse_to_timestamp(args: t.List) -> t.Union[exp.StrToTime, exp.UnixToTime, exp.TimeStrToTime]:
if len(args) == 2: if len(args) == 2:
first_arg, second_arg = args first_arg, second_arg = args
if second_arg.is_string: if second_arg.is_string:
@ -60,8 +60,8 @@ def _parse_to_timestamp(args: t.List) -> t.Union[exp.StrToTime, exp.UnixToTime]:
# reduce it using `simplify_literals` first and then check if it's a Literal. # reduce it using `simplify_literals` first and then check if it's a Literal.
first_arg = seq_get(args, 0) first_arg = seq_get(args, 0)
if not isinstance(simplify_literals(first_arg, root=True), Literal): if not isinstance(simplify_literals(first_arg, root=True), Literal):
# case: <variant_expr> # case: <variant_expr> or other expressions such as columns
return format_time_lambda(exp.StrToTime, "snowflake", default=True)(args) return exp.TimeStrToTime.from_arg_list(args)
if first_arg.is_string: if first_arg.is_string:
if _check_int(first_arg.this): if _check_int(first_arg.this):
@ -560,7 +560,6 @@ class Snowflake(Dialect):
offset = expression.args.get("offset") offset = expression.args.get("offset")
if offset: if offset:
if unnest_alias: if unnest_alias:
expression = expression.copy()
unnest_alias.append("columns", offset.pop()) unnest_alias.append("columns", offset.pop())
selects.append("index") selects.append("index")

2
sqlglot/dialects/spark.py
View file

@ -63,6 +63,8 @@ class Spark(Spark2):
return this return this
class Generator(Spark2.Generator): class Generator(Spark2.Generator):
SUPPORTS_NESTED_CTES = True
TYPE_MAPPING = { TYPE_MAPPING = {
**Spark2.Generator.TYPE_MAPPING, **Spark2.Generator.TYPE_MAPPING,
exp.DataType.Type.MONEY: "DECIMAL(15, 4)", exp.DataType.Type.MONEY: "DECIMAL(15, 4)",

6
sqlglot/dialects/spark2.py
View file

@ -7,7 +7,6 @@ from sqlglot.dialects.dialect import (
binary_from_function, binary_from_function,
format_time_lambda, format_time_lambda,
is_parse_json, is_parse_json,
move_insert_cte_sql,
pivot_column_names, pivot_column_names,
rename_func, rename_func,
trim_sql, trim_sql,
@ -70,7 +69,9 @@ def _unalias_pivot(expression: exp.Expression) -> exp.Expression:
alias = pivot.args["alias"].pop() alias = pivot.args["alias"].pop()
return exp.From( return exp.From(
this=expression.this.replace( this=expression.this.replace(
exp.select("*").from_(expression.this.copy()).subquery(alias=alias) exp.select("*")
.from_(expression.this.copy(), copy=False)
.subquery(alias=alias, copy=False)
) )
) )
@ -188,7 +189,6 @@ class Spark2(Hive):
exp.DayOfYear: rename_func("DAYOFYEAR"), exp.DayOfYear: rename_func("DAYOFYEAR"),
exp.FileFormatProperty: lambda self, e: f"USING {e.name.upper()}", exp.FileFormatProperty: lambda self, e: f"USING {e.name.upper()}",
exp.From: transforms.preprocess([_unalias_pivot]), exp.From: transforms.preprocess([_unalias_pivot]),
exp.Insert: move_insert_cte_sql,
exp.LogicalAnd: rename_func("BOOL_AND"), exp.LogicalAnd: rename_func("BOOL_AND"),
exp.LogicalOr: rename_func("BOOL_OR"), exp.LogicalOr: rename_func("BOOL_OR"),
exp.Map: _map_sql, exp.Map: _map_sql,

2
sqlglot/dialects/sqlite.py
View file

@ -50,7 +50,7 @@ def _transform_create(expression: exp.Expression) -> exp.Expression:
else: else:
for column in defs.values(): for column in defs.values():
auto_increment = None auto_increment = None
for constraint in column.constraints.copy(): for constraint in column.constraints:
if isinstance(constraint.kind, exp.PrimaryKeyColumnConstraint): if isinstance(constraint.kind, exp.PrimaryKeyColumnConstraint):
break break
if isinstance(constraint.kind, exp.AutoIncrementColumnConstraint): if isinstance(constraint.kind, exp.AutoIncrementColumnConstraint):

11
sqlglot/dialects/teradata.py
View file

@ -38,12 +38,15 @@ class Teradata(Dialect):
"^=": TokenType.NEQ, "^=": TokenType.NEQ,
"BYTEINT": TokenType.SMALLINT, "BYTEINT": TokenType.SMALLINT,
"COLLECT": TokenType.COMMAND, "COLLECT": TokenType.COMMAND,
"DEL": TokenType.DELETE,
"EQ": TokenType.EQ,
"GE": TokenType.GTE, "GE": TokenType.GTE,
"GT": TokenType.GT, "GT": TokenType.GT,
"HELP": TokenType.COMMAND, "HELP": TokenType.COMMAND,
"INS": TokenType.INSERT, "INS": TokenType.INSERT,
"LE": TokenType.LTE, "LE": TokenType.LTE,
"LT": TokenType.LT, "LT": TokenType.LT,
"MINUS": TokenType.EXCEPT,
"MOD": TokenType.MOD, "MOD": TokenType.MOD,
"NE": TokenType.NEQ, "NE": TokenType.NEQ,
"NOT=": TokenType.NEQ, "NOT=": TokenType.NEQ,
@ -51,6 +54,7 @@ class Teradata(Dialect):
"SEL": TokenType.SELECT, "SEL": TokenType.SELECT,
"ST_GEOMETRY": TokenType.GEOMETRY, "ST_GEOMETRY": TokenType.GEOMETRY,
"TOP": TokenType.TOP, "TOP": TokenType.TOP,
"UPD": TokenType.UPDATE,
} }
# Teradata does not support % as a modulo operator # Teradata does not support % as a modulo operator
@ -181,6 +185,13 @@ class Teradata(Dialect):
exp.Use: lambda self, e: f"DATABASE {self.sql(e, 'this')}", exp.Use: lambda self, e: f"DATABASE {self.sql(e, 'this')}",
} }
def cast_sql(self, expression: exp.Cast, safe_prefix: t.Optional[str] = None) -> str:
if expression.to.this == exp.DataType.Type.UNKNOWN and expression.args.get("format"):
# We don't actually want to print the unknown type in CAST(<value> AS FORMAT <format>)
expression.to.pop()
return super().cast_sql(expression, safe_prefix=safe_prefix)
def tablesample_sql( def tablesample_sql(
self, expression: exp.TableSample, seed_prefix: str = "SEED", sep=" AS " self, expression: exp.TableSample, seed_prefix: str = "SEED", sep=" AS "
) -> str: ) -> str:

10
sqlglot/dialects/tsql.py
View file

@ -11,7 +11,6 @@ from sqlglot.dialects.dialect import (
generatedasidentitycolumnconstraint_sql, generatedasidentitycolumnconstraint_sql,
max_or_greatest, max_or_greatest,
min_or_least, min_or_least,
move_insert_cte_sql,
parse_date_delta, parse_date_delta,
rename_func, rename_func,
timestrtotime_sql, timestrtotime_sql,
@ -158,8 +157,6 @@ def _format_sql(self: TSQL.Generator, expression: exp.NumberToStr | exp.TimeToSt
def _string_agg_sql(self: TSQL.Generator, expression: exp.GroupConcat) -> str: def _string_agg_sql(self: TSQL.Generator, expression: exp.GroupConcat) -> str:
expression = expression.copy()
this = expression.this this = expression.this
distinct = expression.find(exp.Distinct) distinct = expression.find(exp.Distinct)
if distinct: if distinct:
@ -246,6 +243,7 @@ class TSQL(Dialect):
"MMM": "%b", "MMM": "%b",
"MM": "%m", "MM": "%m",
"M": "%-m", "M": "%-m",
"dddd": "%A",
"dd": "%d", "dd": "%d",
"d": "%-d", "d": "%-d",
"HH": "%H", "HH": "%H",
@ -596,6 +594,8 @@ class TSQL(Dialect):
ALTER_TABLE_ADD_COLUMN_KEYWORD = False ALTER_TABLE_ADD_COLUMN_KEYWORD = False
LIMIT_FETCH = "FETCH" LIMIT_FETCH = "FETCH"
COMPUTED_COLUMN_WITH_TYPE = False COMPUTED_COLUMN_WITH_TYPE = False
SUPPORTS_NESTED_CTES = False
CTE_RECURSIVE_KEYWORD_REQUIRED = False
TYPE_MAPPING = { TYPE_MAPPING = {
**generator.Generator.TYPE_MAPPING, **generator.Generator.TYPE_MAPPING,
@ -622,7 +622,6 @@ class TSQL(Dialect):
exp.GeneratedAsIdentityColumnConstraint: generatedasidentitycolumnconstraint_sql, exp.GeneratedAsIdentityColumnConstraint: generatedasidentitycolumnconstraint_sql,
exp.GroupConcat: _string_agg_sql, exp.GroupConcat: _string_agg_sql,
exp.If: rename_func("IIF"), exp.If: rename_func("IIF"),
exp.Insert: move_insert_cte_sql,
exp.Max: max_or_greatest, exp.Max: max_or_greatest,
exp.MD5: lambda self, e: self.func("HASHBYTES", exp.Literal.string("MD5"), e.this), exp.MD5: lambda self, e: self.func("HASHBYTES", exp.Literal.string("MD5"), e.this),
exp.Min: min_or_least, exp.Min: min_or_least,
@ -685,7 +684,6 @@ class TSQL(Dialect):
return sql return sql
def create_sql(self, expression: exp.Create) -> str: def create_sql(self, expression: exp.Create) -> str:
expression = expression.copy()
kind = self.sql(expression, "kind").upper() kind = self.sql(expression, "kind").upper()
exists = expression.args.pop("exists", None) exists = expression.args.pop("exists", None)
sql = super().create_sql(expression) sql = super().create_sql(expression)
@ -714,7 +712,7 @@ class TSQL(Dialect):
elif expression.args.get("replace"): elif expression.args.get("replace"):
sql = sql.replace("CREATE OR REPLACE ", "CREATE OR ALTER ", 1) sql = sql.replace("CREATE OR REPLACE ", "CREATE OR ALTER ", 1)
return sql return self.prepend_ctes(expression, sql)
def offset_sql(self, expression: exp.Offset) -> str: def offset_sql(self, expression: exp.Offset) -> str:
return f"{super().offset_sql(expression)} ROWS" return f"{super().offset_sql(expression)} ROWS"

86
sqlglot/expressions.py
View file

@ -2145,6 +2145,22 @@ class PartitionedByProperty(Property):
arg_types = {"this": True} arg_types = {"this": True}
# https://www.postgresql.org/docs/current/sql-createtable.html
class PartitionBoundSpec(Expression):
# this -> IN / MODULUS, expression -> REMAINDER, from_expressions -> FROM (...), to_expressions -> TO (...)
arg_types = {
"this": False,
"expression": False,
"from_expressions": False,
"to_expressions": False,
}
class PartitionedOfProperty(Property):
# this -> parent_table (schema), expression -> FOR VALUES ... / DEFAULT
arg_types = {"this": True, "expression": True}
class RemoteWithConnectionModelProperty(Property): class RemoteWithConnectionModelProperty(Property):
arg_types = {"this": True} arg_types = {"this": True}
@ -2486,6 +2502,7 @@ class Table(Expression):
"format": False, "format": False,
"pattern": False, "pattern": False,
"index": False, "index": False,
"ordinality": False,
} }
@property @property
@ -2649,11 +2666,7 @@ class Update(Expression):
class Values(UDTF): class Values(UDTF):
arg_types = { arg_types = {"expressions": True, "alias": False}
"expressions": True,
"ordinality": False,
"alias": False,
}
class Var(Expression): class Var(Expression):
@ -3501,7 +3514,7 @@ class Star(Expression):
class Parameter(Condition): class Parameter(Condition):
arg_types = {"this": True, "wrapped": False} arg_types = {"this": True, "expression": False}
class SessionParameter(Condition): class SessionParameter(Condition):
@ -5036,7 +5049,7 @@ class FromBase(Func):
class Struct(Func): class Struct(Func):
arg_types = {"expressions": True} arg_types = {"expressions": False}
is_var_len_args = True is_var_len_args = True
@ -5171,7 +5184,7 @@ class Use(Expression):
class Merge(Expression): class Merge(Expression):
arg_types = {"this": True, "using": True, "on": True, "expressions": True} arg_types = {"this": True, "using": True, "on": True, "expressions": True, "with": False}
class When(Func): class When(Func):
@ -5459,7 +5472,12 @@ def _wrap(expression: E, kind: t.Type[Expression]) -> E | Paren:
def union( def union(
left: ExpOrStr, right: ExpOrStr, distinct: bool = True, dialect: DialectType = None, **opts left: ExpOrStr,
right: ExpOrStr,
distinct: bool = True,
dialect: DialectType = None,
copy: bool = True,
**opts,
) -> Union: ) -> Union:
""" """
Initializes a syntax tree from one UNION expression. Initializes a syntax tree from one UNION expression.
@ -5475,19 +5493,25 @@ def union(
If an `Expression` instance is passed, it will be used as-is. If an `Expression` instance is passed, it will be used as-is.
distinct: set the DISTINCT flag if and only if this is true. distinct: set the DISTINCT flag if and only if this is true.
dialect: the dialect used to parse the input expression. dialect: the dialect used to parse the input expression.
copy: whether or not to copy the expression.
opts: other options to use to parse the input expressions. opts: other options to use to parse the input expressions.
Returns: Returns:
The new Union instance. The new Union instance.
""" """
left = maybe_parse(sql_or_expression=left, dialect=dialect, **opts) left = maybe_parse(sql_or_expression=left, dialect=dialect, copy=copy, **opts)
right = maybe_parse(sql_or_expression=right, dialect=dialect, **opts) right = maybe_parse(sql_or_expression=right, dialect=dialect, copy=copy, **opts)
return Union(this=left, expression=right, distinct=distinct) return Union(this=left, expression=right, distinct=distinct)
def intersect( def intersect(
left: ExpOrStr, right: ExpOrStr, distinct: bool = True, dialect: DialectType = None, **opts left: ExpOrStr,
right: ExpOrStr,
distinct: bool = True,
dialect: DialectType = None,
copy: bool = True,
**opts,
) -> Intersect: ) -> Intersect:
""" """
Initializes a syntax tree from one INTERSECT expression. Initializes a syntax tree from one INTERSECT expression.
@ -5503,19 +5527,25 @@ def intersect(
If an `Expression` instance is passed, it will be used as-is. If an `Expression` instance is passed, it will be used as-is.
distinct: set the DISTINCT flag if and only if this is true. distinct: set the DISTINCT flag if and only if this is true.
dialect: the dialect used to parse the input expression. dialect: the dialect used to parse the input expression.
copy: whether or not to copy the expression.
opts: other options to use to parse the input expressions. opts: other options to use to parse the input expressions.
Returns: Returns:
The new Intersect instance. The new Intersect instance.
""" """
left = maybe_parse(sql_or_expression=left, dialect=dialect, **opts) left = maybe_parse(sql_or_expression=left, dialect=dialect, copy=copy, **opts)
right = maybe_parse(sql_or_expression=right, dialect=dialect, **opts) right = maybe_parse(sql_or_expression=right, dialect=dialect, copy=copy, **opts)
return Intersect(this=left, expression=right, distinct=distinct) return Intersect(this=left, expression=right, distinct=distinct)
def except_( def except_(
left: ExpOrStr, right: ExpOrStr, distinct: bool = True, dialect: DialectType = None, **opts left: ExpOrStr,
right: ExpOrStr,
distinct: bool = True,
dialect: DialectType = None,
copy: bool = True,
**opts,
) -> Except: ) -> Except:
""" """
Initializes a syntax tree from one EXCEPT expression. Initializes a syntax tree from one EXCEPT expression.
@ -5531,13 +5561,14 @@ def except_(
If an `Expression` instance is passed, it will be used as-is. If an `Expression` instance is passed, it will be used as-is.
distinct: set the DISTINCT flag if and only if this is true. distinct: set the DISTINCT flag if and only if this is true.
dialect: the dialect used to parse the input expression. dialect: the dialect used to parse the input expression.
copy: whether or not to copy the expression.
opts: other options to use to parse the input expressions. opts: other options to use to parse the input expressions.
Returns: Returns:
The new Except instance. The new Except instance.
""" """
left = maybe_parse(sql_or_expression=left, dialect=dialect, **opts) left = maybe_parse(sql_or_expression=left, dialect=dialect, copy=copy, **opts)
right = maybe_parse(sql_or_expression=right, dialect=dialect, **opts) right = maybe_parse(sql_or_expression=right, dialect=dialect, copy=copy, **opts)
return Except(this=left, expression=right, distinct=distinct) return Except(this=left, expression=right, distinct=distinct)
@ -5861,7 +5892,7 @@ def to_identifier(name, quoted=None, copy=True):
Args: Args:
name: The name to turn into an identifier. name: The name to turn into an identifier.
quoted: Whether or not force quote the identifier. quoted: Whether or not force quote the identifier.
copy: Whether or not to copy a passed in Identefier node. copy: Whether or not to copy name if it's an Identifier.
Returns: Returns:
The identifier ast node. The identifier ast node.
@ -5882,6 +5913,25 @@ def to_identifier(name, quoted=None, copy=True):
return identifier return identifier
def parse_identifier(name: str, dialect: DialectType = None) -> Identifier:
"""
Parses a given string into an identifier.
Args:
name: The name to parse into an identifier.
dialect: The dialect to parse against.
Returns:
The identifier ast node.
"""
try:
expression = maybe_parse(name, dialect=dialect, into=Identifier)
except ParseError:
expression = to_identifier(name)
return expression
INTERVAL_STRING_RE = re.compile(r"\s*([0-9]+)\s*([a-zA-Z]+)\s*") INTERVAL_STRING_RE = re.compile(r"\s*([0-9]+)\s*([a-zA-Z]+)\s*")

125
sqlglot/generator.py
View file

@ -230,6 +230,12 @@ class Generator:
# Whether or not data types support additional specifiers like e.g. CHAR or BYTE (oracle) # Whether or not data types support additional specifiers like e.g. CHAR or BYTE (oracle)
DATA_TYPE_SPECIFIERS_ALLOWED = False DATA_TYPE_SPECIFIERS_ALLOWED = False
# Whether or not nested CTEs (e.g. defined inside of subqueries) are allowed
SUPPORTS_NESTED_CTES = True
# Whether or not the "RECURSIVE" keyword is required when defining recursive CTEs
CTE_RECURSIVE_KEYWORD_REQUIRED = True
TYPE_MAPPING = { TYPE_MAPPING = {
exp.DataType.Type.NCHAR: "CHAR", exp.DataType.Type.NCHAR: "CHAR",
exp.DataType.Type.NVARCHAR: "VARCHAR", exp.DataType.Type.NVARCHAR: "VARCHAR",
@ -304,6 +310,7 @@ class Generator:
exp.Order: exp.Properties.Location.POST_SCHEMA, exp.Order: exp.Properties.Location.POST_SCHEMA,
exp.OutputModelProperty: exp.Properties.Location.POST_SCHEMA, exp.OutputModelProperty: exp.Properties.Location.POST_SCHEMA,
exp.PartitionedByProperty: exp.Properties.Location.POST_WITH, exp.PartitionedByProperty: exp.Properties.Location.POST_WITH,
exp.PartitionedOfProperty: exp.Properties.Location.POST_SCHEMA,
exp.PrimaryKey: exp.Properties.Location.POST_SCHEMA, exp.PrimaryKey: exp.Properties.Location.POST_SCHEMA,
exp.Property: exp.Properties.Location.POST_WITH, exp.Property: exp.Properties.Location.POST_WITH,
exp.RemoteWithConnectionModelProperty: exp.Properties.Location.POST_SCHEMA, exp.RemoteWithConnectionModelProperty: exp.Properties.Location.POST_SCHEMA,
@ -407,7 +414,6 @@ class Generator:
"unsupported_messages", "unsupported_messages",
"_escaped_quote_end", "_escaped_quote_end",
"_escaped_identifier_end", "_escaped_identifier_end",
"_cache",
) )
def __init__( def __init__(
@ -447,30 +453,38 @@ class Generator:
self._escaped_identifier_end: str = ( self._escaped_identifier_end: str = (
self.TOKENIZER_CLASS.IDENTIFIER_ESCAPES[0] + self.IDENTIFIER_END self.TOKENIZER_CLASS.IDENTIFIER_ESCAPES[0] + self.IDENTIFIER_END
) )
self._cache: t.Optional[t.Dict[int, str]] = None
def generate( def generate(self, expression: exp.Expression, copy: bool = True) -> str:
self,
expression: t.Optional[exp.Expression],
cache: t.Optional[t.Dict[int, str]] = None,
) -> str:
""" """
Generates the SQL string corresponding to the given syntax tree. Generates the SQL string corresponding to the given syntax tree.
Args: Args:
expression: The syntax tree. expression: The syntax tree.
cache: An optional sql string cache. This leverages the hash of an Expression copy: Whether or not to copy the expression. The generator performs mutations so
which can be slow to compute, so only use it if you set _hash on each node. it is safer to copy.
Returns: Returns:
The SQL string corresponding to `expression`. The SQL string corresponding to `expression`.
""" """
if cache is not None: if copy:
self._cache = cache expression = expression.copy()
# Some dialects only support CTEs at the top level expression, so we need to bubble up nested
# CTEs to that level in order to produce a syntactically valid expression. This transformation
# happens here to minimize code duplication, since many expressions support CTEs.
if (
not self.SUPPORTS_NESTED_CTES
and isinstance(expression, exp.Expression)
and not expression.parent
and "with" in expression.arg_types
and any(node.parent is not expression for node in expression.find_all(exp.With))
):
from sqlglot.transforms import move_ctes_to_top_level
expression = move_ctes_to_top_level(expression)
self.unsupported_messages = [] self.unsupported_messages = []
sql = self.sql(expression).strip() sql = self.sql(expression).strip()
self._cache = None
if self.unsupported_level == ErrorLevel.IGNORE: if self.unsupported_level == ErrorLevel.IGNORE:
return sql return sql
@ -595,12 +609,6 @@ class Generator:
return self.sql(value) return self.sql(value)
return "" return ""
if self._cache is not None:
expression_id = hash(expression)
if expression_id in self._cache:
return self._cache[expression_id]
transform = self.TRANSFORMS.get(expression.__class__) transform = self.TRANSFORMS.get(expression.__class__)
if callable(transform): if callable(transform):
@ -621,11 +629,7 @@ class Generator:
else: else:
raise ValueError(f"Expected an Expression. Received {type(expression)}: {expression}") raise ValueError(f"Expected an Expression. Received {type(expression)}: {expression}")
sql = self.maybe_comment(sql, expression) if self.comments and comment else sql return self.maybe_comment(sql, expression) if self.comments and comment else sql
if self._cache is not None:
self._cache[expression_id] = sql
return sql
def uncache_sql(self, expression: exp.Uncache) -> str: def uncache_sql(self, expression: exp.Uncache) -> str:
table = self.sql(expression, "this") table = self.sql(expression, "this")
@ -879,7 +883,11 @@ class Generator:
def with_sql(self, expression: exp.With) -> str: def with_sql(self, expression: exp.With) -> str:
sql = self.expressions(expression, flat=True) sql = self.expressions(expression, flat=True)
recursive = "RECURSIVE " if expression.args.get("recursive") else "" recursive = (
"RECURSIVE "
if self.CTE_RECURSIVE_KEYWORD_REQUIRED and expression.args.get("recursive")
else ""
)
return f"WITH {recursive}{sql}" return f"WITH {recursive}{sql}"
@ -1022,7 +1030,7 @@ class Generator:
where = self.sql(expression, "expression").strip() where = self.sql(expression, "expression").strip()
return f"{this} FILTER({where})" return f"{this} FILTER({where})"
agg = expression.this.copy() agg = expression.this
agg_arg = agg.this agg_arg = agg.this
cond = expression.expression.this cond = expression.expression.this
agg_arg.replace(exp.If(this=cond.copy(), true=agg_arg.copy())) agg_arg.replace(exp.If(this=cond.copy(), true=agg_arg.copy()))
@ -1088,9 +1096,9 @@ class Generator:
for p in expression.expressions: for p in expression.expressions:
p_loc = self.PROPERTIES_LOCATION[p.__class__] p_loc = self.PROPERTIES_LOCATION[p.__class__]
if p_loc == exp.Properties.Location.POST_WITH: if p_loc == exp.Properties.Location.POST_WITH:
with_properties.append(p.copy()) with_properties.append(p)
elif p_loc == exp.Properties.Location.POST_SCHEMA: elif p_loc == exp.Properties.Location.POST_SCHEMA:
root_properties.append(p.copy()) root_properties.append(p)
return self.root_properties( return self.root_properties(
exp.Properties(expressions=root_properties) exp.Properties(expressions=root_properties)
@ -1124,7 +1132,7 @@ class Generator:
for p in properties.expressions: for p in properties.expressions:
p_loc = self.PROPERTIES_LOCATION[p.__class__] p_loc = self.PROPERTIES_LOCATION[p.__class__]
if p_loc != exp.Properties.Location.UNSUPPORTED: if p_loc != exp.Properties.Location.UNSUPPORTED:
properties_locs[p_loc].append(p.copy()) properties_locs[p_loc].append(p)
else: else:
self.unsupported(f"Unsupported property {p.key}") self.unsupported(f"Unsupported property {p.key}")
@ -1238,6 +1246,29 @@ class Generator:
for_ = " FOR NONE" for_ = " FOR NONE"
return f"WITH{no}{concurrent} ISOLATED LOADING{for_}" return f"WITH{no}{concurrent} ISOLATED LOADING{for_}"
def partitionboundspec_sql(self, expression: exp.PartitionBoundSpec) -> str:
if isinstance(expression.this, list):
return f"IN ({self.expressions(expression, key='this', flat=True)})"
if expression.this:
modulus = self.sql(expression, "this")
remainder = self.sql(expression, "expression")
return f"WITH (MODULUS {modulus}, REMAINDER {remainder})"
from_expressions = self.expressions(expression, key="from_expressions", flat=True)
to_expressions = self.expressions(expression, key="to_expressions", flat=True)
return f"FROM ({from_expressions}) TO ({to_expressions})"
def partitionedofproperty_sql(self, expression: exp.PartitionedOfProperty) -> str:
this = self.sql(expression, "this")
for_values_or_default = expression.expression
if isinstance(for_values_or_default, exp.PartitionBoundSpec):
for_values_or_default = f" FOR VALUES {self.sql(for_values_or_default)}"
else:
for_values_or_default = " DEFAULT"
return f"PARTITION OF {this}{for_values_or_default}"
def lockingproperty_sql(self, expression: exp.LockingProperty) -> str: def lockingproperty_sql(self, expression: exp.LockingProperty) -> str:
kind = expression.args.get("kind") kind = expression.args.get("kind")
this = f" {self.sql(expression, 'this')}" if expression.this else "" this = f" {self.sql(expression, 'this')}" if expression.this else ""
@ -1385,7 +1416,12 @@ class Generator:
index = self.sql(expression, "index") index = self.sql(expression, "index")
index = f" AT {index}" if index else "" index = f" AT {index}" if index else ""
return f"{table}{version}{file_format}{alias}{index}{hints}{pivots}{joins}{laterals}" ordinality = expression.args.get("ordinality") or ""
if ordinality:
ordinality = f" WITH ORDINALITY{alias}"
alias = ""
return f"{table}{version}{file_format}{alias}{index}{hints}{pivots}{joins}{laterals}{ordinality}"
def tablesample_sql( def tablesample_sql(
self, expression: exp.TableSample, seed_prefix: str = "SEED", sep=" AS " self, expression: exp.TableSample, seed_prefix: str = "SEED", sep=" AS "
@ -1489,7 +1525,6 @@ class Generator:
return f"{values} AS {alias}" if alias else values return f"{values} AS {alias}" if alias else values
# Converts `VALUES...` expression into a series of select unions. # Converts `VALUES...` expression into a series of select unions.
expression = expression.copy()
alias_node = expression.args.get("alias") alias_node = expression.args.get("alias")
column_names = alias_node and alias_node.columns column_names = alias_node and alias_node.columns
@ -1972,8 +2007,7 @@ class Generator:
if self.UNNEST_WITH_ORDINALITY: if self.UNNEST_WITH_ORDINALITY:
if alias and isinstance(offset, exp.Expression): if alias and isinstance(offset, exp.Expression):
alias = alias.copy() alias.append("columns", offset)
alias.append("columns", offset.copy())
if alias and self.UNNEST_COLUMN_ONLY: if alias and self.UNNEST_COLUMN_ONLY:
columns = alias.columns columns = alias.columns
@ -2138,7 +2172,6 @@ class Generator:
return f"PRIMARY KEY ({expressions}){options}" return f"PRIMARY KEY ({expressions}){options}"
def if_sql(self, expression: exp.If) -> str: def if_sql(self, expression: exp.If) -> str:
expression = expression.copy()
return self.case_sql(exp.Case(ifs=[expression], default=expression.args.get("false"))) return self.case_sql(exp.Case(ifs=[expression], default=expression.args.get("false")))
def matchagainst_sql(self, expression: exp.MatchAgainst) -> str: def matchagainst_sql(self, expression: exp.MatchAgainst) -> str:
@ -2367,7 +2400,9 @@ class Generator:
def cast_sql(self, expression: exp.Cast, safe_prefix: t.Optional[str] = None) -> str: def cast_sql(self, expression: exp.Cast, safe_prefix: t.Optional[str] = None) -> str:
format_sql = self.sql(expression, "format") format_sql = self.sql(expression, "format")
format_sql = f" FORMAT {format_sql}" if format_sql else "" format_sql = f" FORMAT {format_sql}" if format_sql else ""
return f"{safe_prefix or ''}CAST({self.sql(expression, 'this')} AS {self.sql(expression, 'to')}{format_sql})" to_sql = self.sql(expression, "to")
to_sql = f" {to_sql}" if to_sql else ""
return f"{safe_prefix or ''}CAST({self.sql(expression, 'this')} AS{to_sql}{format_sql})"
def currentdate_sql(self, expression: exp.CurrentDate) -> str: def currentdate_sql(self, expression: exp.CurrentDate) -> str:
zone = self.sql(expression, "this") zone = self.sql(expression, "this")
@ -2510,7 +2545,7 @@ class Generator:
def intdiv_sql(self, expression: exp.IntDiv) -> str: def intdiv_sql(self, expression: exp.IntDiv) -> str:
return self.sql( return self.sql(
exp.Cast( exp.Cast(
this=exp.Div(this=expression.this.copy(), expression=expression.expression.copy()), this=exp.Div(this=expression.this, expression=expression.expression),
to=exp.DataType(this=exp.DataType.Type.INT), to=exp.DataType(this=exp.DataType.Type.INT),
) )
) )
@ -2779,7 +2814,6 @@ class Generator:
hints = table.args.get("hints") hints = table.args.get("hints")
if hints and table.alias and isinstance(hints[0], exp.WithTableHint): if hints and table.alias and isinstance(hints[0], exp.WithTableHint):
# T-SQL syntax is MERGE ... <target_table> [WITH (<merge_hint>)] [[AS] table_alias] # T-SQL syntax is MERGE ... <target_table> [WITH (<merge_hint>)] [[AS] table_alias]
table = table.copy()
table_alias = f" AS {self.sql(table.args['alias'].pop())}" table_alias = f" AS {self.sql(table.args['alias'].pop())}"
this = self.sql(table) this = self.sql(table)
@ -2787,7 +2821,9 @@ class Generator:
on = f"ON {self.sql(expression, 'on')}" on = f"ON {self.sql(expression, 'on')}"
expressions = self.expressions(expression, sep=" ") expressions = self.expressions(expression, sep=" ")
return f"MERGE INTO {this}{table_alias} {using} {on} {expressions}" return self.prepend_ctes(
expression, f"MERGE INTO {this}{table_alias} {using} {on} {expressions}"
)
def tochar_sql(self, expression: exp.ToChar) -> str: def tochar_sql(self, expression: exp.ToChar) -> str:
if expression.args.get("format"): if expression.args.get("format"):
@ -2896,12 +2932,12 @@ class Generator:
case = exp.Case().when( case = exp.Case().when(
expression.this.is_(exp.null()).not_(copy=False), expression.this.is_(exp.null()).not_(copy=False),
expression.args["true"].copy(), expression.args["true"],
copy=False, copy=False,
) )
else_cond = expression.args.get("false") else_cond = expression.args.get("false")
if else_cond: if else_cond:
case.else_(else_cond.copy(), copy=False) case.else_(else_cond, copy=False)
return self.sql(case) return self.sql(case)
@ -2931,15 +2967,6 @@ class Generator:
if not isinstance(expression, exp.Literal): if not isinstance(expression, exp.Literal):
from sqlglot.optimizer.simplify import simplify from sqlglot.optimizer.simplify import simplify
expression = simplify(expression.copy()) expression = simplify(expression)
return expression return expression
def cached_generator(
cache: t.Optional[t.Dict[int, str]] = None
) -> t.Callable[[exp.Expression], str]:
"""Returns a cached generator."""
cache = {} if cache is None else cache
generator = Generator(normalize=True, identify="safe")
return lambda e: generator.generate(e, cache)

4
sqlglot/helper.py
View file

@ -184,9 +184,7 @@ def apply_index_offset(
annotate_types(expression) annotate_types(expression)
if t.cast(exp.DataType, expression.type).this in exp.DataType.INTEGER_TYPES: if t.cast(exp.DataType, expression.type).this in exp.DataType.INTEGER_TYPES:
logger.warning("Applying array index offset (%s)", offset) logger.warning("Applying array index offset (%s)", offset)
expression = simplify( expression = simplify(exp.Add(this=expression, expression=exp.Literal.number(offset)))
exp.Add(this=expression.copy(), expression=exp.Literal.number(offset))
)
return [expression] return [expression]
return expressions return expressions

27
sqlglot/optimizer/normalize.py
View file

@ -4,7 +4,6 @@ import logging
from sqlglot import exp from sqlglot import exp
from sqlglot.errors import OptimizeError from sqlglot.errors import OptimizeError
from sqlglot.generator import cached_generator
from sqlglot.helper import while_changing from sqlglot.helper import while_changing
from sqlglot.optimizer.scope import find_all_in_scope from sqlglot.optimizer.scope import find_all_in_scope
from sqlglot.optimizer.simplify import flatten, rewrite_between, uniq_sort from sqlglot.optimizer.simplify import flatten, rewrite_between, uniq_sort
@ -29,8 +28,6 @@ def normalize(expression: exp.Expression, dnf: bool = False, max_distance: int =
Returns: Returns:
sqlglot.Expression: normalized expression sqlglot.Expression: normalized expression
""" """
generate = cached_generator()
for node, *_ in tuple(expression.walk(prune=lambda e, *_: isinstance(e, exp.Connector))): for node, *_ in tuple(expression.walk(prune=lambda e, *_: isinstance(e, exp.Connector))):
if isinstance(node, exp.Connector): if isinstance(node, exp.Connector):
if normalized(node, dnf=dnf): if normalized(node, dnf=dnf):
@ -49,7 +46,7 @@ def normalize(expression: exp.Expression, dnf: bool = False, max_distance: int =
try: try:
node = node.replace( node = node.replace(
while_changing(node, lambda e: distributive_law(e, dnf, max_distance, generate)) while_changing(node, lambda e: distributive_law(e, dnf, max_distance))
) )
except OptimizeError as e: except OptimizeError as e:
logger.info(e) logger.info(e)
@ -133,7 +130,7 @@ def _predicate_lengths(expression, dnf):
return _predicate_lengths(left, dnf) + _predicate_lengths(right, dnf) return _predicate_lengths(left, dnf) + _predicate_lengths(right, dnf)
def distributive_law(expression, dnf, max_distance, generate): def distributive_law(expression, dnf, max_distance):
""" """
x OR (y AND z) -> (x OR y) AND (x OR z) x OR (y AND z) -> (x OR y) AND (x OR z)
(x AND y) OR (y AND z) -> (x OR y) AND (x OR z) AND (y OR y) AND (y OR z) (x AND y) OR (y AND z) -> (x OR y) AND (x OR z) AND (y OR y) AND (y OR z)
@ -146,7 +143,7 @@ def distributive_law(expression, dnf, max_distance, generate):
if distance > max_distance: if distance > max_distance:
raise OptimizeError(f"Normalization distance {distance} exceeds max {max_distance}") raise OptimizeError(f"Normalization distance {distance} exceeds max {max_distance}")
exp.replace_children(expression, lambda e: distributive_law(e, dnf, max_distance, generate)) exp.replace_children(expression, lambda e: distributive_law(e, dnf, max_distance))
to_exp, from_exp = (exp.Or, exp.And) if dnf else (exp.And, exp.Or) to_exp, from_exp = (exp.Or, exp.And) if dnf else (exp.And, exp.Or)
if isinstance(expression, from_exp): if isinstance(expression, from_exp):
@ -157,30 +154,30 @@ def distributive_law(expression, dnf, max_distance, generate):
if isinstance(a, to_exp) and isinstance(b, to_exp): if isinstance(a, to_exp) and isinstance(b, to_exp):
if len(tuple(a.find_all(exp.Connector))) > len(tuple(b.find_all(exp.Connector))): if len(tuple(a.find_all(exp.Connector))) > len(tuple(b.find_all(exp.Connector))):
return _distribute(a, b, from_func, to_func, generate) return _distribute(a, b, from_func, to_func)
return _distribute(b, a, from_func, to_func, generate) return _distribute(b, a, from_func, to_func)
if isinstance(a, to_exp): if isinstance(a, to_exp):
return _distribute(b, a, from_func, to_func, generate) return _distribute(b, a, from_func, to_func)
if isinstance(b, to_exp): if isinstance(b, to_exp):
return _distribute(a, b, from_func, to_func, generate) return _distribute(a, b, from_func, to_func)
return expression return expression
def _distribute(a, b, from_func, to_func, generate): def _distribute(a, b, from_func, to_func):
if isinstance(a, exp.Connector): if isinstance(a, exp.Connector):
exp.replace_children( exp.replace_children(
a, a,
lambda c: to_func( lambda c: to_func(
uniq_sort(flatten(from_func(c, b.left)), generate), uniq_sort(flatten(from_func(c, b.left))),
uniq_sort(flatten(from_func(c, b.right)), generate), uniq_sort(flatten(from_func(c, b.right))),
copy=False, copy=False,
), ),
) )
else: else:
a = to_func( a = to_func(
uniq_sort(flatten(from_func(a, b.left)), generate), uniq_sort(flatten(from_func(a, b.left))),
uniq_sort(flatten(from_func(a, b.right)), generate), uniq_sort(flatten(from_func(a, b.right))),
copy=False, copy=False,
) )

4
sqlglot/optimizer/normalize_identifiers.py
View file

@ -2,7 +2,7 @@ from __future__ import annotations
import typing as t import typing as t
from sqlglot import exp, parse_one from sqlglot import exp
from sqlglot._typing import E from sqlglot._typing import E
from sqlglot.dialects.dialect import Dialect, DialectType from sqlglot.dialects.dialect import Dialect, DialectType
@ -49,7 +49,7 @@ def normalize_identifiers(expression, dialect=None):
The transformed expression. The transformed expression.
""" """
if isinstance(expression, str): if isinstance(expression, str):
expression = parse_one(expression, dialect=dialect, into=exp.Identifier) expression = exp.parse_identifier(expression, dialect=dialect)
dialect = Dialect.get_or_raise(dialect) dialect = Dialect.get_or_raise(dialect)

2
sqlglot/optimizer/qualify_tables.py
View file

@ -62,7 +62,7 @@ def qualify_tables(
if isinstance(source.this, exp.Identifier): if isinstance(source.this, exp.Identifier):
if not source.args.get("db"): if not source.args.get("db"):
source.set("db", exp.to_identifier(db)) source.set("db", exp.to_identifier(db))
if not source.args.get("catalog"): if not source.args.get("catalog") and source.args.get("db"):
source.set("catalog", exp.to_identifier(catalog)) source.set("catalog", exp.to_identifier(catalog))
if not source.alias: if not source.alias:

77
sqlglot/optimizer/simplify.py
View file

@ -7,8 +7,7 @@ from decimal import Decimal
import sqlglot import sqlglot
from sqlglot import exp from sqlglot import exp
from sqlglot.generator import cached_generator from sqlglot.helper import first, is_iterable, merge_ranges, while_changing
from sqlglot.helper import first, merge_ranges, while_changing
from sqlglot.optimizer.scope import find_all_in_scope, walk_in_scope from sqlglot.optimizer.scope import find_all_in_scope, walk_in_scope
# Final means that an expression should not be simplified # Final means that an expression should not be simplified
@ -37,8 +36,6 @@ def simplify(expression, constant_propagation=False):
sqlglot.Expression: simplified expression sqlglot.Expression: simplified expression
""" """
generate = cached_generator()
# group by expressions cannot be simplified, for example # group by expressions cannot be simplified, for example
# select x + 1 + 1 FROM y GROUP BY x + 1 + 1 # select x + 1 + 1 FROM y GROUP BY x + 1 + 1
# the projection must exactly match the group by key # the projection must exactly match the group by key
@ -67,7 +64,7 @@ def simplify(expression, constant_propagation=False):
# Pre-order transformations # Pre-order transformations
node = expression node = expression
node = rewrite_between(node) node = rewrite_between(node)
node = uniq_sort(node, generate, root) node = uniq_sort(node, root)
node = absorb_and_eliminate(node, root) node = absorb_and_eliminate(node, root)
node = simplify_concat(node) node = simplify_concat(node)
node = simplify_conditionals(node) node = simplify_conditionals(node)
@ -311,7 +308,7 @@ def remove_complements(expression, root=True):
return expression return expression
def uniq_sort(expression, generate, root=True): def uniq_sort(expression, root=True):
""" """
Uniq and sort a connector. Uniq and sort a connector.
@ -320,7 +317,7 @@ def uniq_sort(expression, generate, root=True):
if isinstance(expression, exp.Connector) and (root or not expression.same_parent): if isinstance(expression, exp.Connector) and (root or not expression.same_parent):
result_func = exp.and_ if isinstance(expression, exp.And) else exp.or_ result_func = exp.and_ if isinstance(expression, exp.And) else exp.or_
flattened = tuple(expression.flatten()) flattened = tuple(expression.flatten())
deduped = {generate(e): e for e in flattened} deduped = {gen(e): e for e in flattened}
arr = tuple(deduped.items()) arr = tuple(deduped.items())
# check if the operands are already sorted, if not sort them # check if the operands are already sorted, if not sort them
@ -1070,3 +1067,69 @@ def _flat_simplify(expression, simplifier, root=True):
lambda a, b: expression.__class__(this=a, expression=b), operands lambda a, b: expression.__class__(this=a, expression=b), operands
) )
return expression return expression
def gen(expression: t.Any) -> str:
"""Simple pseudo sql generator for quickly generating sortable and uniq strings.
Sorting and deduping sql is a necessary step for optimization. Calling the actual
generator is expensive so we have a bare minimum sql generator here.
"""
if expression is None:
return "_"
if is_iterable(expression):
return ",".join(gen(e) for e in expression)
if not isinstance(expression, exp.Expression):
return str(expression)
etype = type(expression)
if etype in GEN_MAP:
return GEN_MAP[etype](expression)
return f"{expression.key} {gen(expression.args.values())}"
GEN_MAP = {
exp.Add: lambda e: _binary(e, "+"),
exp.And: lambda e: _binary(e, "AND"),
exp.Anonymous: lambda e: f"{e.this} {','.join(gen(e) for e in e.expressions)}",
exp.Between: lambda e: f"{gen(e.this)} BETWEEN {gen(e.args.get('low'))} AND {gen(e.args.get('high'))}",
exp.Boolean: lambda e: "TRUE" if e.this else "FALSE",
exp.Bracket: lambda e: f"{gen(e.this)}[{gen(e.expressions)}]",
exp.Column: lambda e: ".".join(gen(p) for p in e.parts),
exp.DataType: lambda e: f"{e.this.name} {gen(tuple(e.args.values())[1:])}",
exp.Div: lambda e: _binary(e, "/"),
exp.Dot: lambda e: _binary(e, "."),
exp.DPipe: lambda e: _binary(e, "||"),
exp.SafeDPipe: lambda e: _binary(e, "||"),
exp.EQ: lambda e: _binary(e, "="),
exp.GT: lambda e: _binary(e, ">"),
exp.GTE: lambda e: _binary(e, ">="),
exp.Identifier: lambda e: f'"{e.name}"' if e.quoted else e.name,
exp.ILike: lambda e: _binary(e, "ILIKE"),
exp.In: lambda e: f"{gen(e.this)} IN ({gen(tuple(e.args.values())[1:])})",
exp.Is: lambda e: _binary(e, "IS"),
exp.Like: lambda e: _binary(e, "LIKE"),
exp.Literal: lambda e: f"'{e.name}'" if e.is_string else e.name,
exp.LT: lambda e: _binary(e, "<"),
exp.LTE: lambda e: _binary(e, "<="),
exp.Mod: lambda e: _binary(e, "%"),
exp.Mul: lambda e: _binary(e, "*"),
exp.Neg: lambda e: _unary(e, "-"),
exp.NEQ: lambda e: _binary(e, "<>"),
exp.Not: lambda e: _unary(e, "NOT"),
exp.Null: lambda e: "NULL",
exp.Or: lambda e: _binary(e, "OR"),
exp.Paren: lambda e: f"({gen(e.this)})",
exp.Sub: lambda e: _binary(e, "-"),
exp.Subquery: lambda e: f"({gen(e.args.values())})",
exp.Table: lambda e: gen(e.args.values()),
exp.Var: lambda e: e.name,
}
def _binary(e: exp.Binary, op: str) -> str:
return f"{gen(e.left)} {op} {gen(e.right)}"
def _unary(e: exp.Unary, op: str) -> str:
return f"{op} {gen(e.this)}"

89
sqlglot/parser.py
View file

@ -674,6 +674,7 @@ class Parser(metaclass=_Parser):
"ON": lambda self: self._parse_on_property(), "ON": lambda self: self._parse_on_property(),
"ORDER BY": lambda self: self._parse_order(skip_order_token=True), "ORDER BY": lambda self: self._parse_order(skip_order_token=True),
"OUTPUT": lambda self: self.expression(exp.OutputModelProperty, this=self._parse_schema()), "OUTPUT": lambda self: self.expression(exp.OutputModelProperty, this=self._parse_schema()),
"PARTITION": lambda self: self._parse_partitioned_of(),
"PARTITION BY": lambda self: self._parse_partitioned_by(), "PARTITION BY": lambda self: self._parse_partitioned_by(),
"PARTITIONED BY": lambda self: self._parse_partitioned_by(), "PARTITIONED BY": lambda self: self._parse_partitioned_by(),
"PARTITIONED_BY": lambda self: self._parse_partitioned_by(), "PARTITIONED_BY": lambda self: self._parse_partitioned_by(),
@ -1743,6 +1744,58 @@ class Parser(metaclass=_Parser):
return self._parse_csv(self._parse_conjunction) return self._parse_csv(self._parse_conjunction)
return [] return []
def _parse_partition_bound_spec(self) -> exp.PartitionBoundSpec:
def _parse_partition_bound_expr() -> t.Optional[exp.Expression]:
if self._match_text_seq("MINVALUE"):
return exp.var("MINVALUE")
if self._match_text_seq("MAXVALUE"):
return exp.var("MAXVALUE")
return self._parse_bitwise()
this: t.Optional[exp.Expression | t.List[exp.Expression]] = None
expression = None
from_expressions = None
to_expressions = None
if self._match(TokenType.IN):
this = self._parse_wrapped_csv(self._parse_bitwise)
elif self._match(TokenType.FROM):
from_expressions = self._parse_wrapped_csv(_parse_partition_bound_expr)
self._match_text_seq("TO")
to_expressions = self._parse_wrapped_csv(_parse_partition_bound_expr)
elif self._match_text_seq("WITH", "(", "MODULUS"):
this = self._parse_number()
self._match_text_seq(",", "REMAINDER")
expression = self._parse_number()
self._match_r_paren()
else:
self.raise_error("Failed to parse partition bound spec.")
return self.expression(
exp.PartitionBoundSpec,
this=this,
expression=expression,
from_expressions=from_expressions,
to_expressions=to_expressions,
)
# https://www.postgresql.org/docs/current/sql-createtable.html
def _parse_partitioned_of(self) -> t.Optional[exp.PartitionedOfProperty]:
if not self._match_text_seq("OF"):
self._retreat(self._index - 1)
return None
this = self._parse_table(schema=True)
if self._match(TokenType.DEFAULT):
expression: exp.Var | exp.PartitionBoundSpec = exp.var("DEFAULT")
elif self._match_text_seq("FOR", "VALUES"):
expression = self._parse_partition_bound_spec()
else:
self.raise_error("Expecting either DEFAULT or FOR VALUES clause.")
return self.expression(exp.PartitionedOfProperty, this=this, expression=expression)
def _parse_partitioned_by(self) -> exp.PartitionedByProperty: def _parse_partitioned_by(self) -> exp.PartitionedByProperty:
self._match(TokenType.EQ) self._match(TokenType.EQ)
return self.expression( return self.expression(
@ -2682,6 +2735,10 @@ class Parser(metaclass=_Parser):
for join in iter(self._parse_join, None): for join in iter(self._parse_join, None):
this.append("joins", join) this.append("joins", join)
if self._match_pair(TokenType.WITH, TokenType.ORDINALITY):
this.set("ordinality", True)
this.set("alias", self._parse_table_alias())
return this return this
def _parse_version(self) -> t.Optional[exp.Version]: def _parse_version(self) -> t.Optional[exp.Version]:
@ -4189,17 +4246,12 @@ class Parser(metaclass=_Parser):
fmt = None fmt = None
to = self._parse_types() to = self._parse_types()
if not to: if self._match(TokenType.FORMAT):
self.raise_error("Expected TYPE after CAST")
elif isinstance(to, exp.Identifier):
to = exp.DataType.build(to.name, udt=True)
elif to.this == exp.DataType.Type.CHAR:
if self._match(TokenType.CHARACTER_SET):
to = self.expression(exp.CharacterSet, this=self._parse_var_or_string())
elif self._match(TokenType.FORMAT):
fmt_string = self._parse_string() fmt_string = self._parse_string()
fmt = self._parse_at_time_zone(fmt_string) fmt = self._parse_at_time_zone(fmt_string)
if not to:
to = exp.DataType.build(exp.DataType.Type.UNKNOWN)
if to.this in exp.DataType.TEMPORAL_TYPES: if to.this in exp.DataType.TEMPORAL_TYPES:
this = self.expression( this = self.expression(
exp.StrToDate if to.this == exp.DataType.Type.DATE else exp.StrToTime, exp.StrToDate if to.this == exp.DataType.Type.DATE else exp.StrToTime,
@ -4215,8 +4267,14 @@ class Parser(metaclass=_Parser):
if isinstance(fmt, exp.AtTimeZone) and isinstance(this, exp.StrToTime): if isinstance(fmt, exp.AtTimeZone) and isinstance(this, exp.StrToTime):
this.set("zone", fmt.args["zone"]) this.set("zone", fmt.args["zone"])
return this return this
elif not to:
self.raise_error("Expected TYPE after CAST")
elif isinstance(to, exp.Identifier):
to = exp.DataType.build(to.name, udt=True)
elif to.this == exp.DataType.Type.CHAR:
if self._match(TokenType.CHARACTER_SET):
to = self.expression(exp.CharacterSet, this=self._parse_var_or_string())
return self.expression( return self.expression(
exp.Cast if strict else exp.TryCast, this=this, to=to, format=fmt, safe=safe exp.Cast if strict else exp.TryCast, this=this, to=to, format=fmt, safe=safe
@ -4789,10 +4847,17 @@ class Parser(metaclass=_Parser):
return self._parse_placeholder() return self._parse_placeholder()
def _parse_parameter(self) -> exp.Parameter: def _parse_parameter(self) -> exp.Parameter:
wrapped = self._match(TokenType.L_BRACE) def _parse_parameter_part() -> t.Optional[exp.Expression]:
this = self._parse_var() or self._parse_identifier() or self._parse_primary() return (
self._parse_identifier() or self._parse_primary() or self._parse_var(any_token=True)
)
self._match(TokenType.L_BRACE)
this = _parse_parameter_part()
expression = self._match(TokenType.COLON) and _parse_parameter_part()
self._match(TokenType.R_BRACE) self._match(TokenType.R_BRACE)
return self.expression(exp.Parameter, this=this, wrapped=wrapped)
return self.expression(exp.Parameter, this=this, expression=expression)
def _parse_placeholder(self) -> t.Optional[exp.Expression]: def _parse_placeholder(self) -> t.Optional[exp.Expression]:
if self._match_set(self.PLACEHOLDER_PARSERS): if self._match_set(self.PLACEHOLDER_PARSERS):

14
sqlglot/schema.py
View file

@ -3,10 +3,9 @@ from __future__ import annotations
import abc import abc
import typing as t import typing as t
import sqlglot
from sqlglot import expressions as exp from sqlglot import expressions as exp
from sqlglot.dialects.dialect import Dialect from sqlglot.dialects.dialect import Dialect
from sqlglot.errors import ParseError, SchemaError from sqlglot.errors import SchemaError
from sqlglot.helper import dict_depth from sqlglot.helper import dict_depth
from sqlglot.trie import TrieResult, in_trie, new_trie from sqlglot.trie import TrieResult, in_trie, new_trie
@ -448,19 +447,16 @@ class MappingSchema(AbstractMappingSchema, Schema):
def normalize_name( def normalize_name(
name: str | exp.Identifier, identifier: str | exp.Identifier,
dialect: DialectType = None, dialect: DialectType = None,
is_table: bool = False, is_table: bool = False,
normalize: t.Optional[bool] = True, normalize: t.Optional[bool] = True,
) -> str: ) -> str:
try: if isinstance(identifier, str):
identifier = sqlglot.maybe_parse(name, dialect=dialect, into=exp.Identifier) identifier = exp.parse_identifier(identifier, dialect=dialect)
except ParseError:
return name if isinstance(name, str) else name.name
name = identifier.name
if not normalize: if not normalize:
return name return identifier.name
# This can be useful for normalize_identifier # This can be useful for normalize_identifier
identifier.meta["is_table"] = is_table identifier.meta["is_table"] = is_table

70
sqlglot/transforms.py
View file

@ -67,7 +67,7 @@ def eliminate_distinct_on(expression: exp.Expression) -> exp.Expression:
order = expression.args.get("order") order = expression.args.get("order")
if order: if order:
window.set("order", order.pop().copy()) window.set("order", order.pop())
else: else:
window.set("order", exp.Order(expressions=[c.copy() for c in distinct_cols])) window.set("order", exp.Order(expressions=[c.copy() for c in distinct_cols]))
@ -75,9 +75,9 @@ def eliminate_distinct_on(expression: exp.Expression) -> exp.Expression:
expression.select(window, copy=False) expression.select(window, copy=False)
return ( return (
exp.select(*outer_selects) exp.select(*outer_selects, copy=False)
.from_(expression.subquery("_t")) .from_(expression.subquery("_t", copy=False), copy=False)
.where(exp.column(row_number).eq(1)) .where(exp.column(row_number).eq(1), copy=False)
) )
return expression return expression
@ -120,7 +120,9 @@ def eliminate_qualify(expression: exp.Expression) -> exp.Expression:
elif expr.name not in expression.named_selects: elif expr.name not in expression.named_selects:
expression.select(expr.copy(), copy=False) expression.select(expr.copy(), copy=False)
return outer_selects.from_(expression.subquery(alias="_t")).where(qualify_filters) return outer_selects.from_(expression.subquery(alias="_t", copy=False), copy=False).where(
qualify_filters, copy=False
)
return expression return expression
@ -189,7 +191,7 @@ def explode_to_unnest(index_offset: int = 0) -> t.Callable[[exp.Expression], exp
) )
# we use list here because expression.selects is mutated inside the loop # we use list here because expression.selects is mutated inside the loop
for select in expression.selects.copy(): for select in list(expression.selects):
explode = select.find(exp.Explode) explode = select.find(exp.Explode)
if explode: if explode:
@ -374,6 +376,60 @@ def eliminate_semi_and_anti_joins(expression: exp.Expression) -> exp.Expression:
return expression return expression
def eliminate_full_outer_join(expression: exp.Expression) -> exp.Expression:
"""
Converts a query with a FULL OUTER join to a union of identical queries that
use LEFT/RIGHT OUTER joins instead. This transformation currently only works
for queries that have a single FULL OUTER join.
"""
if isinstance(expression, exp.Select):
full_outer_joins = [
(index, join)
for index, join in enumerate(expression.args.get("joins") or [])
if join.side == "FULL" and join.kind == "OUTER"
]
if len(full_outer_joins) == 1:
expression_copy = expression.copy()
index, full_outer_join = full_outer_joins[0]
full_outer_join.set("side", "left")
expression_copy.args["joins"][index].set("side", "right")
return exp.union(expression, expression_copy, copy=False)
return expression
def move_ctes_to_top_level(expression: exp.Expression) -> exp.Expression:
"""
Some dialects (e.g. Hive, T-SQL, Spark prior to version 3) only allow CTEs to be
defined at the top-level, so for example queries like:
SELECT * FROM (WITH t(c) AS (SELECT 1) SELECT * FROM t) AS subq
are invalid in those dialects. This transformation can be used to ensure all CTEs are
moved to the top level so that the final SQL code is valid from a syntax standpoint.
TODO: handle name clashes whilst moving CTEs (it can get quite tricky & costly).
"""
top_level_with = expression.args.get("with")
for node in expression.find_all(exp.With):
if node.parent is expression:
continue
inner_with = node.pop()
if not top_level_with:
top_level_with = inner_with
expression.set("with", top_level_with)
else:
if inner_with.recursive:
top_level_with.set("recursive", True)
top_level_with.expressions.extend(inner_with.expressions)
return expression
def preprocess( def preprocess(
transforms: t.List[t.Callable[[exp.Expression], exp.Expression]], transforms: t.List[t.Callable[[exp.Expression], exp.Expression]],
) -> t.Callable[[Generator, exp.Expression], str]: ) -> t.Callable[[Generator, exp.Expression], str]:
@ -392,7 +448,7 @@ def preprocess(
def _to_sql(self, expression: exp.Expression) -> str: def _to_sql(self, expression: exp.Expression) -> str:
expression_type = type(expression) expression_type = type(expression)
expression = transforms[0](expression.copy()) expression = transforms[0](expression)
for t in transforms[1:]: for t in transforms[1:]:
expression = t(expression) expression = t(expression)

42
tests/dialects/test_dialect.py
View file

@ -839,10 +839,6 @@ class TestDialect(Validator):
) )
self.validate_all( self.validate_all(
"STR_TO_DATE(x, '%Y-%m-%dT%H:%M:%S')", "STR_TO_DATE(x, '%Y-%m-%dT%H:%M:%S')",
read={
"mysql": "STR_TO_DATE(x, '%Y-%m-%dT%H:%i:%S')",
"starrocks": "STR_TO_DATE(x, '%Y-%m-%dT%H:%i:%S')",
},
write={ write={
"drill": "TO_DATE(x, 'yyyy-MM-dd''T''HH:mm:ss')", "drill": "TO_DATE(x, 'yyyy-MM-dd''T''HH:mm:ss')",
"mysql": "STR_TO_DATE(x, '%Y-%m-%dT%T')", "mysql": "STR_TO_DATE(x, '%Y-%m-%dT%T')",
@ -1861,3 +1857,41 @@ SELECT
"tsql": "SELECT * FROM (SELECT *, COUNT(*) OVER () AS _w FROM t) AS _t WHERE _w > 1", "tsql": "SELECT * FROM (SELECT *, COUNT(*) OVER () AS _w FROM t) AS _t WHERE _w > 1",
}, },
) )
def test_nested_ctes(self):
self.validate_all(
"SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
write={
"bigquery": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"clickhouse": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"databricks": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"duckdb": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"hive": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT c FROM t) AS subq",
"mysql": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"postgres": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"presto": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"redshift": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"snowflake": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"spark": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"spark2": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT c FROM t) AS subq",
"sqlite": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"trino": "SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq",
"tsql": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT c FROM t) AS subq",
},
)
self.validate_all(
"SELECT * FROM (SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq1) AS subq2",
write={
"bigquery": "SELECT * FROM (SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq1) AS subq2",
"duckdb": "SELECT * FROM (SELECT * FROM (WITH t AS (SELECT 1 AS c) SELECT c FROM t) AS subq1) AS subq2",
"hive": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT * FROM (SELECT c FROM t) AS subq1) AS subq2",
"tsql": "WITH t AS (SELECT 1 AS c) SELECT * FROM (SELECT * FROM (SELECT c FROM t) AS subq1) AS subq2",
},
)
self.validate_all(
"WITH t1(x) AS (SELECT 1) SELECT * FROM (WITH t2(y) AS (SELECT 2) SELECT y FROM t2) AS subq",
write={
"duckdb": "WITH t1(x) AS (SELECT 1) SELECT * FROM (WITH t2(y) AS (SELECT 2) SELECT y FROM t2) AS subq",
"tsql": "WITH t1(x) AS (SELECT 1), t2(y) AS (SELECT 2) SELECT * FROM (SELECT y FROM t2) AS subq",
},
)

15
tests/dialects/test_hive.py
View file

@ -429,6 +429,21 @@ class TestHive(Validator):
self.validate_identity( self.validate_identity(
"SELECT key, value, GROUPING__ID, COUNT(*) FROM T1 GROUP BY key, value WITH ROLLUP" "SELECT key, value, GROUPING__ID, COUNT(*) FROM T1 GROUP BY key, value WITH ROLLUP"
) )
self.validate_all(
"SET hiveconf:some_var = 5",
write={
"hive": "SET hiveconf:some_var = 5",
"spark": "SET hiveconf:some_var = 5",
},
)
self.validate_all(
"SELECT ${hiveconf:some_var}",
write={
"hive": "SELECT ${hiveconf:some_var}",
"spark": "SELECT ${hiveconf:some_var}",
},
)
self.validate_all( self.validate_all(
"SELECT A.1a AS b FROM test_a AS A", "SELECT A.1a AS b FROM test_a AS A",
write={ write={

14
tests/dialects/test_mysql.py
View file

@ -564,8 +564,22 @@ class TestMySQL(Validator):
"STR_TO_DATE(x, '%M')", "STR_TO_DATE(x, '%M')",
read={"": "TS_OR_DS_TO_DATE(x, '%B')"}, read={"": "TS_OR_DS_TO_DATE(x, '%B')"},
) )
self.validate_all(
"STR_TO_DATE(x, '%Y-%m-%d')",
write={"presto": "CAST(DATE_PARSE(x, '%Y-%m-%d') AS DATE)"},
)
self.validate_all(
"STR_TO_DATE(x, '%Y-%m-%dT%T')", write={"presto": "DATE_PARSE(x, '%Y-%m-%dT%T')"}
)
def test_mysql(self): def test_mysql(self):
self.validate_all(
# MySQL doesn't support FULL OUTER joins
"SELECT * FROM t1 LEFT OUTER JOIN t2 ON t1.x = t2.x UNION SELECT * FROM t1 RIGHT OUTER JOIN t2 ON t1.x = t2.x",
read={
"postgres": "SELECT * FROM t1 FULL OUTER JOIN t2 ON t1.x = t2.x",
},
)
self.validate_all( self.validate_all(
"a XOR b", "a XOR b",
read={ read={

45
tests/dialects/test_postgres.py
View file

@ -9,22 +9,39 @@ class TestPostgres(Validator):
dialect = "postgres" dialect = "postgres"
def test_ddl(self): def test_ddl(self):
self.validate_identity(
"CREATE INDEX foo ON bar.baz USING btree(col1 varchar_pattern_ops ASC, col2)"
)
self.validate_identity(
"CREATE TABLE test (x TIMESTAMP WITHOUT TIME ZONE[][])",
"CREATE TABLE test (x TIMESTAMP[][])",
)
self.validate_identity("CREATE INDEX idx_x ON x USING BTREE(x, y) WHERE (NOT y IS NULL)") self.validate_identity("CREATE INDEX idx_x ON x USING BTREE(x, y) WHERE (NOT y IS NULL)")
self.validate_identity("CREATE TABLE test (elems JSONB[])") self.validate_identity("CREATE TABLE test (elems JSONB[])")
self.validate_identity("CREATE TABLE public.y (x TSTZRANGE NOT NULL)") self.validate_identity("CREATE TABLE public.y (x TSTZRANGE NOT NULL)")
self.validate_identity("CREATE TABLE test (foo HSTORE)") self.validate_identity("CREATE TABLE test (foo HSTORE)")
self.validate_identity("CREATE TABLE test (foo JSONB)") self.validate_identity("CREATE TABLE test (foo JSONB)")
self.validate_identity("CREATE TABLE test (foo VARCHAR(64)[])") self.validate_identity("CREATE TABLE test (foo VARCHAR(64)[])")
self.validate_identity("CREATE TABLE test (foo INT) PARTITION BY HASH(foo)")
self.validate_identity("INSERT INTO x VALUES (1, 'a', 2.0) RETURNING a") self.validate_identity("INSERT INTO x VALUES (1, 'a', 2.0) RETURNING a")
self.validate_identity("INSERT INTO x VALUES (1, 'a', 2.0) RETURNING a, b") self.validate_identity("INSERT INTO x VALUES (1, 'a', 2.0) RETURNING a, b")
self.validate_identity("INSERT INTO x VALUES (1, 'a', 2.0) RETURNING *") self.validate_identity("INSERT INTO x VALUES (1, 'a', 2.0) RETURNING *")
self.validate_identity("UPDATE tbl_name SET foo = 123 RETURNING a")
self.validate_identity("CREATE TABLE cities_partdef PARTITION OF cities DEFAULT")
self.validate_identity(
"CREATE TABLE cust_part3 PARTITION OF customers FOR VALUES WITH (MODULUS 3, REMAINDER 2)"
)
self.validate_identity(
"CREATE TABLE measurement_y2016m07 PARTITION OF measurement (unitsales DEFAULT 0) FOR VALUES FROM ('2016-07-01') TO ('2016-08-01')"
)
self.validate_identity(
"CREATE TABLE measurement_ym_older PARTITION OF measurement_year_month FOR VALUES FROM (MINVALUE, MINVALUE) TO (2016, 11)"
)
self.validate_identity(
"CREATE TABLE measurement_ym_y2016m11 PARTITION OF measurement_year_month FOR VALUES FROM (2016, 11) TO (2016, 12)"
)
self.validate_identity(
"CREATE TABLE cities_ab PARTITION OF cities (CONSTRAINT city_id_nonzero CHECK (city_id <> 0)) FOR VALUES IN ('a', 'b')"
)
self.validate_identity(
"CREATE TABLE cities_ab PARTITION OF cities (CONSTRAINT city_id_nonzero CHECK (city_id <> 0)) FOR VALUES IN ('a', 'b') PARTITION BY RANGE(population)"
)
self.validate_identity(
"CREATE INDEX foo ON bar.baz USING btree(col1 varchar_pattern_ops ASC, col2)"
)
self.validate_identity( self.validate_identity(
"INSERT INTO x VALUES (1, 'a', 2.0) ON CONFLICT (id) DO NOTHING RETURNING *" "INSERT INTO x VALUES (1, 'a', 2.0) ON CONFLICT (id) DO NOTHING RETURNING *"
) )
@ -43,7 +60,10 @@ class TestPostgres(Validator):
self.validate_identity( self.validate_identity(
"DELETE FROM event USING sales AS s WHERE event.eventid = s.eventid RETURNING a" "DELETE FROM event USING sales AS s WHERE event.eventid = s.eventid RETURNING a"
) )
self.validate_identity("UPDATE tbl_name SET foo = 123 RETURNING a") self.validate_identity(
"CREATE TABLE test (x TIMESTAMP WITHOUT TIME ZONE[][])",
"CREATE TABLE test (x TIMESTAMP[][])",
)
self.validate_all( self.validate_all(
"CREATE OR REPLACE FUNCTION function_name (input_a character varying DEFAULT NULL::character varying)", "CREATE OR REPLACE FUNCTION function_name (input_a character varying DEFAULT NULL::character varying)",
@ -187,6 +207,15 @@ class TestPostgres(Validator):
self.validate_identity( self.validate_identity(
"""ALTER TABLE ONLY "Album" ADD CONSTRAINT "FK_AlbumArtistId" FOREIGN KEY ("ArtistId") REFERENCES "Artist" ("ArtistId") ON DELETE RESTRICT""" """ALTER TABLE ONLY "Album" ADD CONSTRAINT "FK_AlbumArtistId" FOREIGN KEY ("ArtistId") REFERENCES "Artist" ("ArtistId") ON DELETE RESTRICT"""
) )
self.validate_identity(
"SELECT * FROM JSON_ARRAY_ELEMENTS('[1,true, [2,false]]') WITH ORDINALITY"
)
self.validate_identity(
"SELECT * FROM JSON_ARRAY_ELEMENTS('[1,true, [2,false]]') WITH ORDINALITY AS kv_json"
)
self.validate_identity(
"SELECT * FROM JSON_ARRAY_ELEMENTS('[1,true, [2,false]]') WITH ORDINALITY AS kv_json(a, b)"
)
self.validate_identity("x @@ y") self.validate_identity("x @@ y")
self.validate_identity("CAST(x AS MONEY)") self.validate_identity("CAST(x AS MONEY)")
self.validate_identity("CAST(x AS INT4RANGE)") self.validate_identity("CAST(x AS INT4RANGE)")

16
tests/dialects/test_redshift.py
View file

@ -6,10 +6,26 @@ class TestRedshift(Validator):
dialect = "redshift" dialect = "redshift"
def test_redshift(self): def test_redshift(self):
self.validate_identity(
"SELECT DATE_DIFF('month', CAST('2020-02-29 00:00:00' AS TIMESTAMP), CAST('2020-03-02 00:00:00' AS TIMESTAMP))",
"SELECT DATEDIFF(month, CAST(CAST('2020-02-29 00:00:00' AS TIMESTAMP) AS DATE), CAST(CAST('2020-03-02 00:00:00' AS TIMESTAMP) AS DATE))",
)
self.validate_identity( self.validate_identity(
"SELECT * FROM x WHERE y = DATEADD('month', -1, DATE_TRUNC('month', (SELECT y FROM #temp_table)))", "SELECT * FROM x WHERE y = DATEADD('month', -1, DATE_TRUNC('month', (SELECT y FROM #temp_table)))",
"SELECT * FROM x WHERE y = DATEADD(month, -1, CAST(DATE_TRUNC('month', (SELECT y FROM #temp_table)) AS DATE))", "SELECT * FROM x WHERE y = DATEADD(month, -1, CAST(DATE_TRUNC('month', (SELECT y FROM #temp_table)) AS DATE))",
) )
self.validate_all(
"LISTAGG(sellerid, ', ')",
read={
"duckdb": "STRING_AGG(sellerid, ', ')",
},
write={
# GROUP_CONCAT and STRING_AGG are aliases in DuckDB
"duckdb": "GROUP_CONCAT(sellerid, ', ')",
"redshift": "LISTAGG(sellerid, ', ')",
},
)
self.validate_all( self.validate_all(
"SELECT APPROXIMATE COUNT(DISTINCT y)", "SELECT APPROXIMATE COUNT(DISTINCT y)",
read={ read={

5
tests/dialects/test_snowflake.py
View file

@ -13,6 +13,7 @@ class TestSnowflake(Validator):
expr.selects[0].assert_is(exp.AggFunc) expr.selects[0].assert_is(exp.AggFunc)
self.assertEqual(expr.sql(dialect="snowflake"), "SELECT APPROX_TOP_K(C4, 3, 5) FROM t") self.assertEqual(expr.sql(dialect="snowflake"), "SELECT APPROX_TOP_K(C4, 3, 5) FROM t")
self.validate_identity("SELECT OBJECT_CONSTRUCT()")
self.validate_identity("SELECT DAYOFMONTH(CURRENT_TIMESTAMP())") self.validate_identity("SELECT DAYOFMONTH(CURRENT_TIMESTAMP())")
self.validate_identity("SELECT DAYOFYEAR(CURRENT_TIMESTAMP())") self.validate_identity("SELECT DAYOFYEAR(CURRENT_TIMESTAMP())")
self.validate_identity("LISTAGG(data['some_field'], ',')") self.validate_identity("LISTAGG(data['some_field'], ',')")
@ -77,6 +78,10 @@ class TestSnowflake(Validator):
"SELECT {fn CEILING(5.3)}", "SELECT {fn CEILING(5.3)}",
"SELECT CEIL(5.3)", "SELECT CEIL(5.3)",
) )
self.validate_identity(
"SELECT TO_TIMESTAMP(x) FROM t",
"SELECT CAST(x AS TIMESTAMPNTZ) FROM t",
)
self.validate_all("CAST(x AS BYTEINT)", write={"snowflake": "CAST(x AS INT)"}) self.validate_all("CAST(x AS BYTEINT)", write={"snowflake": "CAST(x AS INT)"})
self.validate_all("CAST(x AS CHAR VARYING)", write={"snowflake": "CAST(x AS VARCHAR)"}) self.validate_all("CAST(x AS CHAR VARYING)", write={"snowflake": "CAST(x AS VARCHAR)"})

5
tests/dialects/test_spark.py
View file

@ -592,9 +592,10 @@ TBLPROPERTIES (
self.validate_all( self.validate_all(
"INSERT OVERWRITE TABLE table WITH cte AS (SELECT cola FROM other_table) SELECT cola FROM cte", "INSERT OVERWRITE TABLE table WITH cte AS (SELECT cola FROM other_table) SELECT cola FROM cte",
write={ write={
"spark": "WITH cte AS (SELECT cola FROM other_table) INSERT OVERWRITE TABLE table SELECT cola FROM cte", "databricks": "INSERT OVERWRITE TABLE table WITH cte AS (SELECT cola FROM other_table) SELECT cola FROM cte",
"hive": "WITH cte AS (SELECT cola FROM other_table) INSERT OVERWRITE TABLE table SELECT cola FROM cte",
"spark": "INSERT OVERWRITE TABLE table WITH cte AS (SELECT cola FROM other_table) SELECT cola FROM cte",
"spark2": "WITH cte AS (SELECT cola FROM other_table) INSERT OVERWRITE TABLE table SELECT cola FROM cte", "spark2": "WITH cte AS (SELECT cola FROM other_table) INSERT OVERWRITE TABLE table SELECT cola FROM cte",
"databricks": "WITH cte AS (SELECT cola FROM other_table) INSERT OVERWRITE TABLE table SELECT cola FROM cte",
}, },
) )

26
tests/dialects/test_teradata.py
View file

@ -148,18 +148,21 @@ class TestTeradata(Validator):
self.validate_all("a MOD b", write={"teradata": "a MOD b", "mysql": "a % b"}) self.validate_all("a MOD b", write={"teradata": "a MOD b", "mysql": "a % b"})
def test_abbrev(self): def test_abbrev(self):
self.validate_all("a LT b", write={"teradata": "a < b"}) self.validate_identity("a LT b", "a < b")
self.validate_all("a LE b", write={"teradata": "a <= b"}) self.validate_identity("a LE b", "a <= b")
self.validate_all("a GT b", write={"teradata": "a > b"}) self.validate_identity("a GT b", "a > b")
self.validate_all("a GE b", write={"teradata": "a >= b"}) self.validate_identity("a GE b", "a >= b")
self.validate_all("a ^= b", write={"teradata": "a <> b"}) self.validate_identity("a ^= b", "a <> b")
self.validate_all("a NE b", write={"teradata": "a <> b"}) self.validate_identity("a NE b", "a <> b")
self.validate_all("a NOT= b", write={"teradata": "a <> b"}) self.validate_identity("a NOT= b", "a <> b")
self.validate_identity("a EQ b", "a = b")
self.validate_all( self.validate_identity("SEL a FROM b", "SELECT a FROM b")
"SEL a FROM b", self.validate_identity(
write={"teradata": "SELECT a FROM b"}, "SELECT col1, col2 FROM dbc.table1 WHERE col1 EQ 'value1' MINUS SELECT col1, col2 FROM dbc.table2",
"SELECT col1, col2 FROM dbc.table1 WHERE col1 = 'value1' EXCEPT SELECT col1, col2 FROM dbc.table2",
) )
self.validate_identity("UPD a SET b = 1", "UPDATE a SET b = 1")
self.validate_identity("DEL FROM a", "DELETE FROM a")
def test_datatype(self): def test_datatype(self):
self.validate_all( self.validate_all(
@ -187,3 +190,4 @@ class TestTeradata(Validator):
"": "STR_TO_DATE('1992-01', '%Y-%d')", "": "STR_TO_DATE('1992-01', '%Y-%d')",
}, },
) )
self.validate_identity("CAST('1992-01' AS FORMAT 'YYYY-DD')")

44
tests/dialects/test_tsql.py
View file

@ -6,6 +6,36 @@ class TestTSQL(Validator):
dialect = "tsql" dialect = "tsql"
def test_tsql(self): def test_tsql(self):
self.validate_all(
"WITH t(c) AS (SELECT 1) SELECT * INTO foo FROM (SELECT c FROM t) AS temp",
read={
"duckdb": "CREATE TABLE foo AS WITH t(c) AS (SELECT 1) SELECT c FROM t",
},
)
self.validate_all(
"WITH y AS (SELECT 2 AS c) INSERT INTO t SELECT * FROM y",
read={
"duckdb": "WITH y AS (SELECT 2 AS c) INSERT INTO t SELECT * FROM y",
},
)
self.validate_all(
"WITH t(c) AS (SELECT 1) SELECT 1 AS c UNION (SELECT c FROM t)",
read={
"duckdb": "SELECT 1 AS c UNION (WITH t(c) AS (SELECT 1) SELECT c FROM t)",
},
)
self.validate_all(
"WITH t(c) AS (SELECT 1) MERGE INTO x AS z USING (SELECT c FROM t) AS y ON a = b WHEN MATCHED THEN UPDATE SET a = y.b",
read={
"postgres": "MERGE INTO x AS z USING (WITH t(c) AS (SELECT 1) SELECT c FROM t) AS y ON a = b WHEN MATCHED THEN UPDATE SET a = y.b",
},
)
self.validate_all(
"WITH t(n) AS (SELECT 1 AS n UNION ALL SELECT n + 1 AS n FROM t WHERE n < 4) SELECT * FROM (SELECT SUM(n) AS s4 FROM t) AS subq",
read={
"duckdb": "SELECT * FROM (WITH RECURSIVE t(n) AS (SELECT 1 AS n UNION ALL SELECT n + 1 AS n FROM t WHERE n < 4) SELECT SUM(n) AS s4 FROM t) AS subq",
},
)
self.validate_all( self.validate_all(
"CREATE TABLE #mytemptable (a INTEGER)", "CREATE TABLE #mytemptable (a INTEGER)",
read={ read={
@ -825,12 +855,18 @@ WHERE
def test_datename(self): def test_datename(self):
self.validate_all( self.validate_all(
"SELECT DATENAME(mm,'1970-01-01')", "SELECT DATENAME(mm, '1970-01-01')",
write={"spark": "SELECT DATE_FORMAT(CAST('1970-01-01' AS TIMESTAMP), 'MMMM')"}, write={
"spark": "SELECT DATE_FORMAT(CAST('1970-01-01' AS TIMESTAMP), 'MMMM')",
"tsql": "SELECT FORMAT(CAST('1970-01-01' AS DATETIME2), 'MMMM')",
},
) )
self.validate_all( self.validate_all(
"SELECT DATENAME(dw,'1970-01-01')", "SELECT DATENAME(dw, '1970-01-01')",
write={"spark": "SELECT DATE_FORMAT(CAST('1970-01-01' AS TIMESTAMP), 'EEEE')"}, write={
"spark": "SELECT DATE_FORMAT(CAST('1970-01-01' AS TIMESTAMP), 'EEEE')",
"tsql": "SELECT FORMAT(CAST('1970-01-01' AS DATETIME2), 'dddd')",
},
) )
def test_datepart(self): def test_datepart(self):

2
tests/fixtures/optimizer/normalize.sql vendored
View file

@ -44,4 +44,4 @@ SELECT * FROM x WHERE (A AND B) OR C;
SELECT * FROM x WHERE (A OR C) AND (B OR C); SELECT * FROM x WHERE (A OR C) AND (B OR C);
dt2 between '2022-01-01 12:00:00' and '2022-12-31' and dt2 >= '2022-05-01 12:00:00' or dt2 = '2021-06-01 12:00:00'; dt2 between '2022-01-01 12:00:00' and '2022-12-31' and dt2 >= '2022-05-01 12:00:00' or dt2 = '2021-06-01 12:00:00';
(dt2 <= '2022-12-31' OR dt2 = '2021-06-01 12:00:00') AND (dt2 = '2021-06-01 12:00:00' OR dt2 >= '2022-01-01 12:00:00') AND (dt2 = '2021-06-01 12:00:00' OR dt2 >= '2022-05-01 12:00:00') (dt2 <= '2022-12-31' OR dt2 = '2021-06-01 12:00:00') AND (dt2 = '2021-06-01 12:00:00' OR dt2 >= '2022-01-01 12:00:00') AND (dt2 = '2021-06-01 12:00:00' OR dt2 >= '2022-05-01 12:00:00');

3
tests/fixtures/optimizer/normalize_identifiers.sql vendored
View file

@ -70,3 +70,6 @@ SELECT a /* sqlglot.meta case_sensitive */, B FROM table /* sqlglot.meta case_se
# dialect: redshift # dialect: redshift
SELECT COALESCE(json_val.a /* sqlglot.meta case_sensitive */, json_val.A /* sqlglot.meta case_sensitive */) FROM table; SELECT COALESCE(json_val.a /* sqlglot.meta case_sensitive */, json_val.A /* sqlglot.meta case_sensitive */) FROM table;
SELECT COALESCE(json_val.a /* sqlglot.meta case_sensitive */, json_val.A /* sqlglot.meta case_sensitive */) FROM table; SELECT COALESCE(json_val.a /* sqlglot.meta case_sensitive */, json_val.A /* sqlglot.meta case_sensitive */) FROM table;
SELECT @X;
SELECT @X;

4
tests/fixtures/optimizer/optimizer.sql vendored
View file

@ -480,8 +480,8 @@ JOIN "company_table" AS "company_table_2"
LEFT JOIN "unlocked" AS "unlocked" LEFT JOIN "unlocked" AS "unlocked"
ON "company_table_2"."id" = "unlocked"."company_id" ON "company_table_2"."id" = "unlocked"."company_id"
WHERE WHERE
CASE WHEN "unlocked"."company_id" IS NULL THEN 0 ELSE 1 END = FALSE NOT "company_table_2"."id" IS NULL
AND NOT "company_table_2"."id" IS NULL; AND CASE WHEN "unlocked"."company_id" IS NULL THEN 0 ELSE 1 END = FALSE;
# title: db.table alias clash # title: db.table alias clash
# execute: false # execute: false

6
tests/fixtures/optimizer/pushdown_predicates.sql vendored
View file

@ -11,7 +11,7 @@ SELECT x.a FROM (SELECT * FROM x) AS x JOIN y WHERE (x.a = y.a AND x.a = 1 AND x
SELECT x.a FROM (SELECT * FROM x) AS x JOIN y ON x.a = y.a WHERE TRUE; SELECT x.a FROM (SELECT * FROM x) AS x JOIN y ON x.a = y.a WHERE TRUE;
SELECT x.a FROM (SELECT * FROM x) AS x JOIN y WHERE (x.a = y.a AND x.a = 1 AND x.b = 1) OR x.a = y.b; SELECT x.a FROM (SELECT * FROM x) AS x JOIN y WHERE (x.a = y.a AND x.a = 1 AND x.b = 1) OR x.a = y.b;
SELECT x.a FROM (SELECT * FROM x) AS x JOIN y ON x.a = y.a OR x.a = y.b WHERE (x.a = y.a AND x.a = 1 AND x.b = 1) OR x.a = y.b; SELECT x.a FROM (SELECT * FROM x) AS x JOIN y ON x.a = y.a OR x.a = y.b WHERE (x.a = 1 AND x.a = y.a AND x.b = 1) OR x.a = y.b;
SELECT x.a FROM (SELECT x.a AS a, x.b * 1 AS c FROM x) AS x WHERE x.c = 1; SELECT x.a FROM (SELECT x.a AS a, x.b * 1 AS c FROM x) AS x WHERE x.c = 1;
SELECT x.a FROM (SELECT x.a AS a, x.b * 1 AS c FROM x WHERE x.b * 1 = 1) AS x WHERE TRUE; SELECT x.a FROM (SELECT x.a AS a, x.b * 1 AS c FROM x WHERE x.b * 1 = 1) AS x WHERE TRUE;
@ -26,10 +26,10 @@ SELECT x.a FROM x AS x JOIN (SELECT y.a FROM y AS y) AS y ON y.a = 1 AND x.a = y
SELECT x.a FROM x AS x JOIN (SELECT y.a FROM y AS y WHERE y.a = 1) AS y ON x.a = y.a AND TRUE; SELECT x.a FROM x AS x JOIN (SELECT y.a FROM y AS y WHERE y.a = 1) AS y ON x.a = y.a AND TRUE;
SELECT x.a AS a FROM x AS x JOIN (SELECT * FROM y AS y) AS y ON y.a = 1 WHERE x.a = 1 AND x.b = 1 AND y.a = x.a; SELECT x.a AS a FROM x AS x JOIN (SELECT * FROM y AS y) AS y ON y.a = 1 WHERE x.a = 1 AND x.b = 1 AND y.a = x.a;
SELECT x.a AS a FROM x AS x JOIN (SELECT * FROM y AS y WHERE y.a = 1) AS y ON y.a = x.a AND TRUE WHERE x.a = 1 AND x.b = 1 AND TRUE; SELECT x.a AS a FROM x AS x JOIN (SELECT * FROM y AS y WHERE y.a = 1) AS y ON TRUE AND y.a = x.a WHERE x.a = 1 AND x.b = 1 AND TRUE;
SELECT x.a AS a FROM x AS x CROSS JOIN (SELECT * FROM y AS y) AS y WHERE x.a = 1 AND x.b = 1 AND y.a = x.a AND y.a = 1; SELECT x.a AS a FROM x AS x CROSS JOIN (SELECT * FROM y AS y) AS y WHERE x.a = 1 AND x.b = 1 AND y.a = x.a AND y.a = 1;
SELECT x.a AS a FROM x AS x JOIN (SELECT * FROM y AS y WHERE y.a = 1) AS y ON y.a = x.a AND TRUE WHERE x.a = 1 AND x.b = 1 AND TRUE AND TRUE; SELECT x.a AS a FROM x AS x JOIN (SELECT * FROM y AS y WHERE y.a = 1) AS y ON TRUE AND y.a = x.a WHERE x.a = 1 AND x.b = 1 AND TRUE AND TRUE;
with t1 as (SELECT x.a, x.b, ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) as row_num FROM x) SELECT t1.a, t1.b FROM t1 WHERE row_num = 1; with t1 as (SELECT x.a, x.b, ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) as row_num FROM x) SELECT t1.a, t1.b FROM t1 WHERE row_num = 1;
WITH t1 AS (SELECT x.a, x.b, ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) AS row_num FROM x) SELECT t1.a, t1.b FROM t1 WHERE row_num = 1; WITH t1 AS (SELECT x.a, x.b, ROW_NUMBER() OVER (PARTITION BY x.a ORDER BY x.a) AS row_num FROM x) SELECT t1.a, t1.b FROM t1 WHERE row_num = 1;

4
tests/fixtures/optimizer/qualify_tables.sql vendored
View file

@ -117,3 +117,7 @@ SELECT x FROM c.db.t AS t, LATERAL UNNEST(t.xs) AS x;
# title: lateral unnest without alias # title: lateral unnest without alias
SELECT x FROM t, LATERAL UNNEST(t.xs); SELECT x FROM t, LATERAL UNNEST(t.xs);
SELECT x FROM c.db.t AS t, LATERAL UNNEST(t.xs) AS _q_0; SELECT x FROM c.db.t AS t, LATERAL UNNEST(t.xs) AS _q_0;
# title: table with ordinality
SELECT * FROM t CROSS JOIN JSON_ARRAY_ELEMENTS(t.response) WITH ORDINALITY AS kv_json;
SELECT * FROM c.db.t AS t CROSS JOIN JSON_ARRAY_ELEMENTS(t.response) WITH ORDINALITY AS kv_json;

3
tests/fixtures/optimizer/quote_identities.sql vendored
View file

@ -6,3 +6,6 @@ SELECT "a" FROM "x";
SELECT x.a AS a FROM db.x; SELECT x.a AS a FROM db.x;
SELECT "x"."a" AS "a" FROM "db"."x"; SELECT "x"."a" AS "a" FROM "db"."x";
SELECT @x;
SELECT @x;

20
tests/fixtures/optimizer/simplify.sql vendored
View file

@ -631,19 +631,19 @@ COALESCE(x);
x; x;
COALESCE(x, 1) = 2; COALESCE(x, 1) = 2;
x = 2 AND NOT x IS NULL; NOT x IS NULL AND x = 2;
2 = COALESCE(x, 1); 2 = COALESCE(x, 1);
2 = x AND NOT x IS NULL; 2 = x AND NOT x IS NULL;
COALESCE(x, 1, 1) = 1 + 1; COALESCE(x, 1, 1) = 1 + 1;
x = 2 AND NOT x IS NULL; NOT x IS NULL AND x = 2;
COALESCE(x, 1, 2) = 2; COALESCE(x, 1, 2) = 2;
x = 2 AND NOT x IS NULL; NOT x IS NULL AND x = 2;
COALESCE(x, 3) <= 2; COALESCE(x, 3) <= 2;
x <= 2 AND NOT x IS NULL; NOT x IS NULL AND x <= 2;
COALESCE(x, 1) <> 2; COALESCE(x, 1) <> 2;
x <> 2 OR x IS NULL; x <> 2 OR x IS NULL;
@ -872,7 +872,7 @@ x = 5 AND y = x;
x = 5 AND y = 5; x = 5 AND y = 5;
5 = x AND y = x; 5 = x AND y = x;
y = 5 AND 5 = x; 5 = x AND y = 5;
x = 5 OR y = x; x = 5 OR y = x;
x = 5 OR y = x; x = 5 OR y = x;
@ -899,13 +899,13 @@ x = 5 AND x + 3 = 8;
x = 5; x = 5;
x = 5 AND (SELECT x FROM t WHERE y = 1); x = 5 AND (SELECT x FROM t WHERE y = 1);
x = 5 AND (SELECT x FROM t WHERE y = 1); (SELECT x FROM t WHERE y = 1) AND x = 5;
x = 1 AND y > 0 AND (SELECT z = 5 FROM t WHERE y = 1); x = 1 AND y > 0 AND (SELECT z = 5 FROM t WHERE y = 1);
x = 1 AND y > 0 AND (SELECT z = 5 FROM t WHERE y = 1); (SELECT z = 5 FROM t WHERE y = 1) AND x = 1 AND y > 0;
x = 1 AND x = y AND (SELECT z FROM t WHERE a AND (b OR c)); x = 1 AND x = y AND (SELECT z FROM t WHERE a AND (b OR c));
x = 1 AND (SELECT z FROM t WHERE a AND (b OR c)) AND 1 = y; (SELECT z FROM t WHERE a AND (b OR c)) AND 1 = y AND x = 1;
t1.a = 39 AND t2.b = t1.a AND t3.c = t2.b; t1.a = 39 AND t2.b = t1.a AND t3.c = t2.b;
t1.a = 39 AND t2.b = 39 AND t3.c = 39; t1.a = 39 AND t2.b = 39 AND t3.c = 39;
@ -920,10 +920,10 @@ x = 1 AND CASE x WHEN 5 THEN FALSE ELSE TRUE END;
x = 1; x = 1;
x = y AND CASE WHEN x = 5 THEN FALSE ELSE TRUE END; x = y AND CASE WHEN x = 5 THEN FALSE ELSE TRUE END;
x = y AND CASE WHEN x = 5 THEN FALSE ELSE TRUE END; CASE WHEN x = 5 THEN FALSE ELSE TRUE END AND x = y;
x = 1 AND CASE WHEN y = 5 THEN x = z END; x = 1 AND CASE WHEN y = 5 THEN x = z END;
x = 1 AND CASE WHEN y = 5 THEN 1 = z END; CASE WHEN y = 5 THEN 1 = z END AND x = 1;
-------------------------------------- --------------------------------------
-- Simplify Conditionals -- Simplify Conditionals

4
tests/fixtures/optimizer/tpc-ds/tpc-ds.sql vendored
View file

@ -2547,8 +2547,8 @@ JOIN "date_dim" AS "date_dim"
) )
WHERE WHERE
"_u_3"."_u_4" IS NULL "_u_3"."_u_4" IS NULL
AND ARRAY_ANY("_u_0"."_u_2", "_x" -> "cs1"."cs_warehouse_sk" <> "_x")
AND NOT "_u_0"."_u_1" IS NULL AND NOT "_u_0"."_u_1" IS NULL
AND ARRAY_ANY("_u_0"."_u_2", "_x" -> "cs1"."cs_warehouse_sk" <> "_x")
ORDER BY ORDER BY
COUNT(DISTINCT "cs1"."cs_order_number") COUNT(DISTINCT "cs1"."cs_order_number")
LIMIT 100; LIMIT 100;
@ -12586,8 +12586,8 @@ JOIN "web_site" AS "web_site"
AND "ws1"."ws_web_site_sk" = "web_site"."web_site_sk" AND "ws1"."ws_web_site_sk" = "web_site"."web_site_sk"
WHERE WHERE
"_u_3"."_u_4" IS NULL "_u_3"."_u_4" IS NULL
AND ARRAY_ANY("_u_0"."_u_2", "_x" -> "ws1"."ws_warehouse_sk" <> "_x")
AND NOT "_u_0"."_u_1" IS NULL AND NOT "_u_0"."_u_1" IS NULL
AND ARRAY_ANY("_u_0"."_u_2", "_x" -> "ws1"."ws_warehouse_sk" <> "_x")
ORDER BY ORDER BY
COUNT(DISTINCT "ws1"."ws_order_number") COUNT(DISTINCT "ws1"."ws_order_number")
LIMIT 100; LIMIT 100;

6
tests/fixtures/optimizer/tpc-h/tpc-h.sql vendored
View file

@ -249,9 +249,9 @@ FROM "orders" AS "orders"
LEFT JOIN "_u_0" AS "_u_0" LEFT JOIN "_u_0" AS "_u_0"
ON "_u_0"."l_orderkey" = "orders"."o_orderkey" ON "_u_0"."l_orderkey" = "orders"."o_orderkey"
WHERE WHERE
CAST("orders"."o_orderdate" AS DATE) < CAST('1993-10-01' AS DATE) NOT "_u_0"."l_orderkey" IS NULL
AND CAST("orders"."o_orderdate" AS DATE) < CAST('1993-10-01' AS DATE)
AND CAST("orders"."o_orderdate" AS DATE) >= CAST('1993-07-01' AS DATE) AND CAST("orders"."o_orderdate" AS DATE) >= CAST('1993-07-01' AS DATE)
AND NOT "_u_0"."l_orderkey" IS NULL
GROUP BY GROUP BY
"orders"."o_orderpriority" "orders"."o_orderpriority"
ORDER BY ORDER BY
@ -1348,8 +1348,8 @@ WHERE
"_u_2"."l_orderkey" IS NULL "_u_2"."l_orderkey" IS NULL
OR NOT ARRAY_ANY("_u_2"."_u_3", "_x" -> "_x" <> "l1"."l_suppkey") OR NOT ARRAY_ANY("_u_2"."_u_3", "_x" -> "_x" <> "l1"."l_suppkey")
) )
AND ARRAY_ANY("_u_0"."_u_1", "_x" -> "_x" <> "l1"."l_suppkey")
AND NOT "_u_0"."l_orderkey" IS NULL AND NOT "_u_0"."l_orderkey" IS NULL
AND ARRAY_ANY("_u_0"."_u_1", "_x" -> "_x" <> "l1"."l_suppkey")
GROUP BY GROUP BY
"supplier"."s_name" "supplier"."s_name"
ORDER BY ORDER BY

2
tests/test_expressions.py
View file

@ -443,7 +443,7 @@ class TestExpressions(unittest.TestCase):
return None return None
return node return node
self.assertEqual(expression.transform(remove_non_list_arg).sql(), "CAST(x AS )") self.assertEqual(expression.transform(remove_non_list_arg).sql(), "CAST(x AS)")
expression = parse_one("SELECT a, b FROM x") expression = parse_one("SELECT a, b FROM x")

18
tests/test_optimizer.py
View file

@ -246,6 +246,15 @@ class TestOptimizer(unittest.TestCase):
"CREATE FUNCTION `udfs`.`myTest`(`x` FLOAT64) AS (1)", "CREATE FUNCTION `udfs`.`myTest`(`x` FLOAT64) AS (1)",
) )
self.assertEqual(
optimizer.qualify.qualify(
parse_one("SELECT `bar_bazfoo_$id` FROM test", read="spark"),
schema={"test": {"bar_bazFoo_$id": "BIGINT"}},
dialect="spark",
).sql(dialect="spark"),
"SELECT `test`.`bar_bazfoo_$id` AS `bar_bazfoo_$id` FROM `test` AS `test`",
)
self.check_file( self.check_file(
"qualify_columns", qualify_columns, execute=True, schema=self.schema, set_dialect=True "qualify_columns", qualify_columns, execute=True, schema=self.schema, set_dialect=True
) )
@ -271,6 +280,8 @@ class TestOptimizer(unittest.TestCase):
set_dialect=True, set_dialect=True,
) )
self.assertEqual(optimizer.normalize_identifiers.normalize_identifiers("a%").sql(), '"a%"')
def test_pushdown_projection(self): def test_pushdown_projection(self):
self.check_file("pushdown_projections", pushdown_projections, schema=self.schema) self.check_file("pushdown_projections", pushdown_projections, schema=self.schema)
@ -978,3 +989,10 @@ FROM READ_CSV('tests/fixtures/optimizer/tpc-h/nation.csv.gz', 'delimiter', '|')
query = parse_one("select a.b:c from d", read="snowflake") query = parse_one("select a.b:c from d", read="snowflake")
qualified = optimizer.qualify.qualify(query) qualified = optimizer.qualify.qualify(query)
self.assertEqual(qualified.expressions[0].alias, "c") self.assertEqual(qualified.expressions[0].alias, "c")
def test_qualify_tables_no_schema(self):
query = parse_one("select a from b")
self.assertEqual(
optimizer.qualify_tables.qualify_tables(query, catalog="catalog").sql(),
"SELECT a FROM b AS b",
)