Adding upstream version 17.7.0.
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
parent
5c63f2bde9
commit
4a22906fbb
GPG key ID:
FBB4F0E80A80222F
93 changed files with 41580 additions and 39040 deletions
50
CHANGELOG.md
50
CHANGELOG.md
|
|
@ -1,6 +1,50 @@
|
|||
Changelog
|
||||
=========
|
||||
|
||||
## [v17.6.1] - 2023-07-21
|
||||
### :sparkles: New Features
|
||||
- [`4b7e9f1`](https://github.com/tobymao/sqlglot/commit/4b7e9f1d2e3de9eb9b96acfd9e6d8566663666c4) - **clickhouse**: add support for the logical xor function *(PR [#1937](https://github.com/tobymao/sqlglot/pull/1937) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- :arrow_lower_right: *addresses issue [#1936](undefined) opened by [@cpcloud](https://github.com/cpcloud)*
|
||||
- [`1d2b5e0`](https://github.com/tobymao/sqlglot/commit/1d2b5e0a8519692cb7ffdb461d2360304349b415) - **hive**: add support for the query TRANSFORM clause *(PR [#1935](https://github.com/tobymao/sqlglot/pull/1935) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- :arrow_lower_right: *addresses issue [#1933](undefined) opened by [@sangongs](https://github.com/sangongs)*
|
||||
|
||||
### :bug: Bug Fixes
|
||||
- [`b8de650`](https://github.com/tobymao/sqlglot/commit/b8de650ca6b9e21e65c29f037dd3c1167e3bf29a) - union lineage with > 2 sources closes [#1934](https://github.com/tobymao/sqlglot/pull/1934) *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
- [`79efb42`](https://github.com/tobymao/sqlglot/commit/79efb42fde0b8c6e0f878b55ca292cac5a535535) - **duckdb, presto**: improve bitwise support *(PR [#1938](https://github.com/tobymao/sqlglot/pull/1938) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
|
||||
|
||||
## [v17.6.0] - 2023-07-19
|
||||
### :sparkles: New Features
|
||||
- [`2510999`](https://github.com/tobymao/sqlglot/commit/2510999192aa98e74d4ea713eb3e7f7d5b4b846f) - isin unnest builder *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
|
||||
|
||||
## [v17.5.0] - 2023-07-18
|
||||
### :sparkles: New Features
|
||||
- [`e6b3a01`](https://github.com/tobymao/sqlglot/commit/e6b3a017da747fb38d298617ca310bb7657711da) - **tsql**: improve support for transaction statements *(PR [#1907](https://github.com/tobymao/sqlglot/pull/1907) by [@dmoore247](https://github.com/dmoore247))*
|
||||
- [`048b9bc`](https://github.com/tobymao/sqlglot/commit/048b9bcd023e6dd6a47fff1536d66e1ca94991f4) - add refresh command *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
- [`3456bbf`](https://github.com/tobymao/sqlglot/commit/3456bbf09ce8adf28fa29ac4c2c7a196ecb0099f) - add RegexpReplace expression *(PR [#1925](https://github.com/tobymao/sqlglot/pull/1925) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- [`d6c8722`](https://github.com/tobymao/sqlglot/commit/d6c8722a1a1d10f0599c7216a668d083edac0fab) - clickhouse array join *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
|
||||
### :bug: Bug Fixes
|
||||
- [`d928ee0`](https://github.com/tobymao/sqlglot/commit/d928ee0e4ced53ec9e4868388f334b03c339e5e0) - duckdb filter where optional *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
- [`cbcb113`](https://github.com/tobymao/sqlglot/commit/cbcb113f9b2a5cf2d46871b0c8d15af383452576) - **spark**: add support for RLIKE function *(PR [#1911](https://github.com/tobymao/sqlglot/pull/1911) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- [`cc33749`](https://github.com/tobymao/sqlglot/commit/cc337493378be5815c04f6c40ee471657f19a485) - preserve comments in exp.Drop *(commit by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- [`35e05f7`](https://github.com/tobymao/sqlglot/commit/35e05f758547f9d249ef77f383ddef698947179b) - make end transactions postgres specific closes [#1921](https://github.com/tobymao/sqlglot/pull/1921) *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
- [`aaee594`](https://github.com/tobymao/sqlglot/commit/aaee59457eca7255540c27a4aa13a7a4f87e6acc) - **teradata**: separate POST_EXPRESSION props from POST_INDEX *(PR [#1924](https://github.com/tobymao/sqlglot/pull/1924) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- :arrow_lower_right: *fixes issue [#1923](undefined) opened by [@MarkBell920](https://github.com/MarkBell920)*
|
||||
- [`a523c8b`](https://github.com/tobymao/sqlglot/commit/a523c8b507c7af393aa178dce4a6e337b9548e8e) - comments above limit *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
- [`e9f1cb5`](https://github.com/tobymao/sqlglot/commit/e9f1cb53b1b73f5698d5cce5728cfc92d2020375) - **spark,duckdb**: transpile TO_TIMESTAMP, MONTHS_BETWEEN correctly *(PR [#1929](https://github.com/tobymao/sqlglot/pull/1929) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- :arrow_lower_right: *fixes issue [#1928](undefined) opened by [@richard-a-lott](https://github.com/richard-a-lott)*
|
||||
|
||||
### :recycle: Refactors
|
||||
- [`726306e`](https://github.com/tobymao/sqlglot/commit/726306e9a77a700555b31fbe136e6438f711178f) - **scope**: rename _is_subquery_scope to _is_derived_table for clarity *(PR [#1912](https://github.com/tobymao/sqlglot/pull/1912) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
|
||||
|
||||
## [v17.4.1] - 2023-07-11
|
||||
### :bug: Bug Fixes
|
||||
- [`461aeaf`](https://github.com/tobymao/sqlglot/commit/461aeaf794cf541cebc1a5cce727907f93d44119) - handle dots in table name *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
|
||||
|
||||
## [v17.4.0] - 2023-07-11
|
||||
### :boom: BREAKING CHANGES
|
||||
- due to [`c511278`](https://github.com/tobymao/sqlglot/commit/c51127842ffbb5dbe02c33de3ba8e04b69e6ebc3) - table_name force quotes if unsafe identifiers *(commit by [@tobymao](https://github.com/tobymao))*:
|
||||
|
|
@ -858,4 +902,8 @@ Changelog
|
|||
[v17.1.0]: https://github.com/tobymao/sqlglot/compare/v17.0.0...v17.1.0
|
||||
[v17.2.0]: https://github.com/tobymao/sqlglot/compare/v17.1.0...v17.2.0
|
||||
[v17.3.0]: https://github.com/tobymao/sqlglot/compare/v17.2.0...v17.3.0
|
||||
[v17.4.0]: https://github.com/tobymao/sqlglot/compare/v17.3.0...v17.4.0
|
||||
[v17.4.0]: https://github.com/tobymao/sqlglot/compare/v17.3.0...v17.4.0
|
||||
[v17.4.1]: https://github.com/tobymao/sqlglot/compare/v17.4.0...v17.4.1
|
||||
[v17.5.0]: https://github.com/tobymao/sqlglot/compare/v17.4.1...v17.5.0
|
||||
[v17.6.0]: https://github.com/tobymao/sqlglot/compare/v17.5.0...v17.6.0
|
||||
[v17.6.1]: https://github.com/tobymao/sqlglot/compare/v17.6.0...v17.6.1
|
||||
|
|
@ -70,7 +70,7 @@ We'd love to hear from you. Join our community [Slack channel](https://tobikodat
|
|||
|
||||
### Formatting and Transpiling
|
||||
|
||||
Easily translate from one dialect to another. For example, date/time functions vary from dialects and can be hard to deal with:
|
||||
Easily translate from one dialect to another. For example, date/time functions vary between dialects and can be hard to deal with:
|
||||
|
||||
```python
|
||||
import sqlglot
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -169,7 +169,7 @@
|
|||
|
||||
<h3 id="formatting-and-transpiling">Formatting and Transpiling</h3>
|
||||
|
||||
<p>Easily translate from one dialect to another. For example, date/time functions vary from dialects and can be hard to deal with:</p>
|
||||
<p>Easily translate from one dialect to another. For example, date/time functions vary between dialects and can be hard to deal with:</p>
|
||||
|
||||
<div class="pdoc-code codehilite">
|
||||
<pre><span></span><code><span class="kn">import</span> <span class="nn">sqlglot</span>
|
||||
|
|
|
|||
|
|
@ -51,8 +51,8 @@
|
|||
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos">1</span></a><span class="c1"># file generated by setuptools_scm</span>
|
||||
</span><span id="L-2"><a href="#L-2"><span class="linenos">2</span></a><span class="c1"># don't change, don't track in version control</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos">3</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">'17.4.0'</span>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos">4</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">17</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos">3</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">'17.6.1'</span>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos">4</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">17</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -777,7 +777,7 @@
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">createDataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">data</span><span class="p">:</span> <span class="n">Sequence</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904735552'</span><span class="o">></span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904735552'</span><span class="o">></span><span class="p">],</span> <span class="n">Tuple</span><span class="p">]]</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411906690064'</span><span class="o">></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">samplingRatio</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">verifySchema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
<span class="name">createDataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">data</span><span class="p">:</span> <span class="n">Sequence</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539402082944'</span><span class="o">></span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539402082944'</span><span class="o">></span><span class="p">],</span> <span class="n">Tuple</span><span class="p">]]</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539402129216'</span><span class="o">></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">samplingRatio</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">verifySchema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="SparkSession.createDataFrame-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -1681,7 +1681,7 @@
|
|||
<input id="DataFrame.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
|
||||
<span class="name">DataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">spark</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411909715808'</span><span class="o">></span>,</span><span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Select">sqlglot.expressions.Select</a></span>,</span><span class="param"> <span class="n">branch_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">sequence_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">last_op</span><span class="p">:</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">dataframe</span><span class="o">.</span><span class="n">sql</span><span class="o">.</span><span class="n">operations</span><span class="o">.</span><span class="n">Operation</span> <span class="o">=</span> <span class="o"><</span><span class="n">Operation</span><span class="o">.</span><span class="n">INIT</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span><span class="o">></span>,</span><span class="param"> <span class="n">pending_hints</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">output_expression_container</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411909885280'</span><span class="o">></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span>)</span>
|
||||
<span class="name">DataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">spark</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539404916848'</span><span class="o">></span>,</span><span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Select">sqlglot.expressions.Select</a></span>,</span><span class="param"> <span class="n">branch_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">sequence_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">last_op</span><span class="p">:</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">dataframe</span><span class="o">.</span><span class="n">sql</span><span class="o">.</span><span class="n">operations</span><span class="o">.</span><span class="n">Operation</span> <span class="o">=</span> <span class="o"><</span><span class="n">Operation</span><span class="o">.</span><span class="n">INIT</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span><span class="o">></span>,</span><span class="param"> <span class="n">pending_hints</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">output_expression_container</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539405009872'</span><span class="o">></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span>)</span>
|
||||
|
||||
<label class="view-source-button" for="DataFrame.__init__-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -2611,7 +2611,7 @@ is unlikely to come up.</p>
|
|||
<div class="decorator">@operation(Operation.FROM)</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">fillna</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411905248896'</span><span class="o">></span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
<span class="name">fillna</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539400523248'</span><span class="o">></span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="DataFrame.fillna-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -2680,7 +2680,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="decorator">@operation(Operation.FROM)</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">replace</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">to_replace</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">Dict</span><span class="p">]</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n">Collection</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903718352'</span><span class="o">></span><span class="p">],</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903718352'</span><span class="o">></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
<span class="name">replace</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">to_replace</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">Dict</span><span class="p">]</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n">Collection</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539400461696'</span><span class="o">></span><span class="p">],</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539400461696'</span><span class="o">></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="DataFrame.replace-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -2885,7 +2885,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="decorator">@operation(Operation.NO_OP)</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">repartition</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">numPartitions</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903922128'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904015376'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
<span class="name">repartition</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">numPartitions</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539400857312'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539401003136'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="DataFrame.repartition-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -3369,7 +3369,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
</span><span id="Column-115"><a href="#Column-115"><span class="linenos">115</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">inverse_binary_op</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">,</span> <span class="n">other</span><span class="p">)</span>
|
||||
</span><span id="Column-116"><a href="#Column-116"><span class="linenos">116</span></a>
|
||||
</span><span id="Column-117"><a href="#Column-117"><span class="linenos">117</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="Column-118"><a href="#Column-118"><span class="linenos">118</span></a> <span class="k">def</span> <span class="nf">ensure_col</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Union</span><span class="p">[</span><span class="n">ColumnOrLiteral</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">]]):</span>
|
||||
</span><span id="Column-118"><a href="#Column-118"><span class="linenos">118</span></a> <span class="k">def</span> <span class="nf">ensure_col</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Union</span><span class="p">[</span><span class="n">ColumnOrLiteral</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">]])</span> <span class="o">-></span> <span class="n">Column</span><span class="p">:</span>
|
||||
</span><span id="Column-119"><a href="#Column-119"><span class="linenos">119</span></a> <span class="k">return</span> <span class="bp">cls</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
|
||||
</span><span id="Column-120"><a href="#Column-120"><span class="linenos">120</span></a>
|
||||
</span><span id="Column-121"><a href="#Column-121"><span class="linenos">121</span></a> <span class="nd">@classmethod</span>
|
||||
|
|
@ -3514,7 +3514,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
</span><span id="Column-260"><a href="#Column-260"><span class="linenos">260</span></a> <span class="n">new_expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">Not</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Is</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">column_expression</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Null</span><span class="p">()))</span>
|
||||
</span><span id="Column-261"><a href="#Column-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="n">Column</span><span class="p">(</span><span class="n">new_expression</span><span class="p">)</span>
|
||||
</span><span id="Column-262"><a href="#Column-262"><span class="linenos">262</span></a>
|
||||
</span><span id="Column-263"><a href="#Column-263"><span class="linenos">263</span></a> <span class="k">def</span> <span class="nf">cast</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dataType</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">DataType</span><span class="p">]):</span>
|
||||
</span><span id="Column-263"><a href="#Column-263"><span class="linenos">263</span></a> <span class="k">def</span> <span class="nf">cast</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dataType</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">DataType</span><span class="p">])</span> <span class="o">-></span> <span class="n">Column</span><span class="p">:</span>
|
||||
</span><span id="Column-264"><a href="#Column-264"><span class="linenos">264</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="Column-265"><a href="#Column-265"><span class="linenos">265</span></a><span class="sd"> Functionality Difference: PySpark cast accepts a datatype instance of the datatype class</span>
|
||||
</span><span id="Column-266"><a href="#Column-266"><span class="linenos">266</span></a><span class="sd"> Sqlglot doesn't currently replicate this class so it only accepts a string</span>
|
||||
|
|
@ -3590,7 +3590,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<input id="Column.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
|
||||
<span class="name">Column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411907807680'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span>)</span>
|
||||
<span class="name">Column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539402936640'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span>)</span>
|
||||
|
||||
<label class="view-source-button" for="Column.__init__-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -3633,14 +3633,14 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">ensure_col</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904110128'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span><span class="return-annotation">):</span></span>
|
||||
<span class="name">ensure_col</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539401082464'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.ensure_col-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#Column.ensure_col"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Column.ensure_col-117"><a href="#Column.ensure_col-117"><span class="linenos">117</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="Column.ensure_col-118"><a href="#Column.ensure_col-118"><span class="linenos">118</span></a> <span class="k">def</span> <span class="nf">ensure_col</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Union</span><span class="p">[</span><span class="n">ColumnOrLiteral</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">]]):</span>
|
||||
</span><span id="Column.ensure_col-118"><a href="#Column.ensure_col-118"><span class="linenos">118</span></a> <span class="k">def</span> <span class="nf">ensure_col</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Union</span><span class="p">[</span><span class="n">ColumnOrLiteral</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">]])</span> <span class="o">-></span> <span class="n">Column</span><span class="p">:</span>
|
||||
</span><span id="Column.ensure_col-119"><a href="#Column.ensure_col-119"><span class="linenos">119</span></a> <span class="k">return</span> <span class="bp">cls</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
|
@ -3654,7 +3654,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">ensure_cols</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">args</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904283728'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span><span class="p">]</span>:</span></span>
|
||||
<span class="name">ensure_cols</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">args</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539401020528'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span><span class="p">]</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.ensure_cols-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -3675,7 +3675,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">invoke_anonymous_function</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903793648'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="n">func_name</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904390720'</span><span class="o">></span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
<span class="name">invoke_anonymous_function</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539401141168'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="n">func_name</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539401263168'</span><span class="o">></span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.invoke_anonymous_function-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -3702,7 +3702,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">invoke_expression_over_column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904276080'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="n">callable_expression</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
<span class="name">invoke_expression_over_column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539399493696'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="n">callable_expression</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.invoke_expression_over_column-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -3739,7 +3739,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904480480'</span><span class="o">></span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
<span class="name">binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539399386160'</span><span class="o">></span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.binary_op-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -3760,7 +3760,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">inverse_binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904268496'</span><span class="o">></span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
<span class="name">inverse_binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539399414416'</span><span class="o">></span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.inverse_binary_op-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -4160,13 +4160,13 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">cast</span><span class="signature pdoc-code condensed">(<span class="param"><span class="bp">self</span>, </span><span class="param"><span class="n">dataType</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">dataframe</span><span class="o">.</span><span class="n">sql</span><span class="o">.</span><span class="n">types</span><span class="o">.</span><span class="n">DataType</span><span class="p">]</span></span><span class="return-annotation">):</span></span>
|
||||
<span class="name">cast</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">dataType</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">dataframe</span><span class="o">.</span><span class="n">sql</span><span class="o">.</span><span class="n">types</span><span class="o">.</span><span class="n">DataType</span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.cast-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#Column.cast"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Column.cast-263"><a href="#Column.cast-263"><span class="linenos">263</span></a> <span class="k">def</span> <span class="nf">cast</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dataType</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">DataType</span><span class="p">]):</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Column.cast-263"><a href="#Column.cast-263"><span class="linenos">263</span></a> <span class="k">def</span> <span class="nf">cast</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dataType</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">DataType</span><span class="p">])</span> <span class="o">-></span> <span class="n">Column</span><span class="p">:</span>
|
||||
</span><span id="Column.cast-264"><a href="#Column.cast-264"><span class="linenos">264</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="Column.cast-265"><a href="#Column.cast-265"><span class="linenos">265</span></a><span class="sd"> Functionality Difference: PySpark cast accepts a datatype instance of the datatype class</span>
|
||||
</span><span id="Column.cast-266"><a href="#Column.cast-266"><span class="linenos">266</span></a><span class="sd"> Sqlglot doesn't currently replicate this class so it only accepts a string</span>
|
||||
|
|
@ -4314,7 +4314,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">isin</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902623952'</span><span class="o">></span><span class="p">,</span> <span class="n">Iterable</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902623952'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">):</span></span>
|
||||
<span class="name">isin</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539399635728'</span><span class="o">></span><span class="p">,</span> <span class="n">Iterable</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539399635728'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">):</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.isin-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -4335,7 +4335,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">between</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">lowerBound</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902714368'</span><span class="o">></span>,</span><span class="param"> <span class="n">upperBound</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902752464'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
<span class="name">between</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">lowerBound</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539399680064'</span><span class="o">></span>,</span><span class="param"> <span class="n">upperBound</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539399772416'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.between-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -4370,7 +4370,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">over</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">window</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902826560'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
<span class="name">over</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">window</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539399831664'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.over-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -4615,7 +4615,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
|||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903155680'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903155680'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539400316976'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539400316976'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Window.partitionBy-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -4636,7 +4636,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
|||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903260272'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903260272'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539400303328'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539400303328'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Window.orderBy-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -4872,7 +4872,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902905984'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902905984'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539400150784'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539400150784'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="WindowSpec.partitionBy-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -4899,7 +4899,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902894160'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902894160'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539400013664'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140539400013664'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="WindowSpec.orderBy-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load diff
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -589,26 +589,26 @@ Default: True</li>
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#Trino.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Trino.Generator.can_identify-246"><a href="#Trino.Generator.can_identify-246"><span class="linenos">246</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="Trino.Generator.can_identify-247"><a href="#Trino.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="Trino.Generator.can_identify-248"><a href="#Trino.Generator.can_identify-248"><span class="linenos">248</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="Trino.Generator.can_identify-249"><a href="#Trino.Generator.can_identify-249"><span class="linenos">249</span></a>
|
||||
</span><span id="Trino.Generator.can_identify-250"><a href="#Trino.Generator.can_identify-250"><span class="linenos">250</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Trino.Generator.can_identify-251"><a href="#Trino.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="Trino.Generator.can_identify-252"><a href="#Trino.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="Trino.Generator.can_identify-253"><a href="#Trino.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="Trino.Generator.can_identify-254"><a href="#Trino.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="Trino.Generator.can_identify-255"><a href="#Trino.Generator.can_identify-255"><span class="linenos">255</span></a>
|
||||
</span><span id="Trino.Generator.can_identify-256"><a href="#Trino.Generator.can_identify-256"><span class="linenos">256</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Trino.Generator.can_identify-257"><a href="#Trino.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="Trino.Generator.can_identify-258"><a href="#Trino.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> """</span>
|
||||
</span><span id="Trino.Generator.can_identify-259"><a href="#Trino.Generator.can_identify-259"><span class="linenos">259</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="Trino.Generator.can_identify-260"><a href="#Trino.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="Trino.Generator.can_identify-261"><a href="#Trino.Generator.can_identify-261"><span class="linenos">261</span></a>
|
||||
</span><span id="Trino.Generator.can_identify-262"><a href="#Trino.Generator.can_identify-262"><span class="linenos">262</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="Trino.Generator.can_identify-263"><a href="#Trino.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="Trino.Generator.can_identify-264"><a href="#Trino.Generator.can_identify-264"><span class="linenos">264</span></a>
|
||||
</span><span id="Trino.Generator.can_identify-265"><a href="#Trino.Generator.can_identify-265"><span class="linenos">265</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Trino.Generator.can_identify-248"><a href="#Trino.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="Trino.Generator.can_identify-249"><a href="#Trino.Generator.can_identify-249"><span class="linenos">249</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="Trino.Generator.can_identify-250"><a href="#Trino.Generator.can_identify-250"><span class="linenos">250</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="Trino.Generator.can_identify-251"><a href="#Trino.Generator.can_identify-251"><span class="linenos">251</span></a>
|
||||
</span><span id="Trino.Generator.can_identify-252"><a href="#Trino.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Trino.Generator.can_identify-253"><a href="#Trino.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="Trino.Generator.can_identify-254"><a href="#Trino.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="Trino.Generator.can_identify-255"><a href="#Trino.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="Trino.Generator.can_identify-256"><a href="#Trino.Generator.can_identify-256"><span class="linenos">256</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="Trino.Generator.can_identify-257"><a href="#Trino.Generator.can_identify-257"><span class="linenos">257</span></a>
|
||||
</span><span id="Trino.Generator.can_identify-258"><a href="#Trino.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Trino.Generator.can_identify-259"><a href="#Trino.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="Trino.Generator.can_identify-260"><a href="#Trino.Generator.can_identify-260"><span class="linenos">260</span></a><span class="sd"> """</span>
|
||||
</span><span id="Trino.Generator.can_identify-261"><a href="#Trino.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="Trino.Generator.can_identify-262"><a href="#Trino.Generator.can_identify-262"><span class="linenos">262</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="Trino.Generator.can_identify-263"><a href="#Trino.Generator.can_identify-263"><span class="linenos">263</span></a>
|
||||
</span><span id="Trino.Generator.can_identify-264"><a href="#Trino.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="Trino.Generator.can_identify-265"><a href="#Trino.Generator.can_identify-265"><span class="linenos">265</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="Trino.Generator.can_identify-266"><a href="#Trino.Generator.can_identify-266"><span class="linenos">266</span></a>
|
||||
</span><span id="Trino.Generator.can_identify-267"><a href="#Trino.Generator.can_identify-267"><span class="linenos">267</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1064,6 +1064,7 @@ Default: True</li>
|
|||
<dd id="Trino.Generator.oncluster_sql" class="function"><a href="../generator.html#Generator.oncluster_sql">oncluster_sql</a></dd>
|
||||
<dd id="Trino.Generator.clusteredbyproperty_sql" class="function"><a href="../generator.html#Generator.clusteredbyproperty_sql">clusteredbyproperty_sql</a></dd>
|
||||
<dd id="Trino.Generator.anyvalue_sql" class="function"><a href="../generator.html#Generator.anyvalue_sql">anyvalue_sql</a></dd>
|
||||
<dd id="Trino.Generator.querytransform_sql" class="function"><a href="../generator.html#Generator.querytransform_sql">querytransform_sql</a></dd>
|
||||
|
||||
</div>
|
||||
<div><dt><a href="presto.html#Presto.Generator">sqlglot.dialects.presto.Presto.Generator</a></dt>
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load diff
|
|
@ -191,7 +191,7 @@
|
|||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="k">raise</span> <span class="n">SqlglotError</span><span class="p">(</span><span class="s2">"Cannot build lineage, sql must be SELECT"</span><span class="p">)</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="k">def</span> <span class="nf">to_node</span><span class="p">(</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="n">column_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="n">column</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">int</span><span class="p">,</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="n">scope</span><span class="p">:</span> <span class="n">Scope</span><span class="p">,</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="n">scope_name</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="n">upstream</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Node</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
|
|
@ -202,177 +202,189 @@
|
|||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="k">for</span> <span class="n">dt</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="k">if</span> <span class="n">dt</span><span class="o">.</span><span class="n">comments</span> <span class="ow">and</span> <span class="n">dt</span><span class="o">.</span><span class="n">comments</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"source: "</span><span class="p">)</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="p">}</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Union</span><span class="p">):</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">union_scopes</span><span class="p">:</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">to_node</span><span class="p">(</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">column_name</span><span class="p">,</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">scope</span><span class="o">=</span><span class="n">scope</span><span class="p">,</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">scope_name</span><span class="o">=</span><span class="n">scope_name</span><span class="p">,</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">upstream</span><span class="o">=</span><span class="n">upstream</span><span class="p">,</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">scope_name</span><span class="p">),</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="p">)</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="c1"># Find the specific select clause that is the source of the column we want.</span>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="c1"># This can either be a specific, named select or a generic `*` clause.</span>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="n">select</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="p">(</span><span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">column_name</span><span class="p">),</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Star</span><span class="p">()</span> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">is_star</span> <span class="k">else</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="p">)</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="c1"># Find the specific select clause that is the source of the column we want.</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="c1"># This can either be a specific, named select or a generic `*` clause.</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">select</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">[</span><span class="n">column</span><span class="p">]</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">int</span><span class="p">)</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="k">else</span> <span class="nb">next</span><span class="p">(</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="p">(</span><span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">column</span><span class="p">),</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Star</span><span class="p">()</span> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">is_star</span> <span class="k">else</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="p">)</span>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="p">)</span>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">select</span><span class="p">:</span>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Could not find </span><span class="si">{</span><span class="n">column</span><span class="si">}</span><span class="s2"> in </span><span class="si">{</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Union</span><span class="p">):</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">upstream</span> <span class="o">=</span> <span class="n">upstream</span> <span class="ow">or</span> <span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">"UNION"</span><span class="p">,</span> <span class="n">source</span><span class="o">=</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">select</span><span class="p">)</span>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">select</span><span class="p">:</span>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Could not find </span><span class="si">{</span><span class="n">column_name</span><span class="si">}</span><span class="s2"> in </span><span class="si">{</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="c1"># For better ergonomics in our node labels, replace the full select with</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="c1"># a version that has only the column we care about.</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="c1"># "x", SELECT x, y FROM foo</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="c1"># => "x", SELECT x FROM foo</span>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">append</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="c1"># Create the node for this step in the lineage chain, and attach it to the previous one.</span>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">Node</span><span class="p">(</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">scope_name</span><span class="si">}</span><span class="s2">.</span><span class="si">{</span><span class="n">column_name</span><span class="si">}</span><span class="s2">"</span> <span class="k">if</span> <span class="n">scope_name</span> <span class="k">else</span> <span class="n">column_name</span><span class="p">,</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">expression</span><span class="o">=</span><span class="n">select</span><span class="p">,</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">alias</span> <span class="ow">or</span> <span class="s2">""</span><span class="p">,</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="p">)</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">if</span> <span class="n">upstream</span><span class="p">:</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">upstream</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="c1"># Find all columns that went into creating this one to list their lineage nodes.</span>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="nb">set</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)):</span>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">c</span><span class="o">.</span><span class="n">table</span>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span>
|
||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="c1"># The table itself came from a more specific scope. Recurse into that one using the unaliased column name.</span>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">to_node</span><span class="p">(</span>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">c</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">scope_name</span><span class="o">=</span><span class="n">table</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">node</span><span class="p">,</span> <span class="n">alias</span><span class="o">=</span><span class="n">aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="p">)</span>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="c1"># The source is not a scope - we've reached the end of the line. At this point, if a source is not found</span>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="c1"># it means this column's lineage is unknown. This can happen if the definition of a source used in a query</span>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="c1"># is not passed into the `sources` map.</span>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">source</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">Placeholder</span><span class="p">()</span>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="n">c</span><span class="o">.</span><span class="n">sql</span><span class="p">(),</span> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">source</span><span class="p">))</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="n">index</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="n">column</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">int</span><span class="p">)</span>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="k">else</span> <span class="nb">next</span><span class="p">(</span>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">i</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">select</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">column</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="p">)</span>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="p">)</span>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">union_scopes</span><span class="p">:</span>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="n">to_node</span><span class="p">(</span><span class="n">index</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">s</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">upstream</span><span class="p">)</span>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">return</span> <span class="n">upstream</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="c1"># For better ergonomics in our node labels, replace the full select with</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="c1"># a version that has only the column we care about.</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="c1"># "x", SELECT x, y FROM foo</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="c1"># => "x", SELECT x FROM foo</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">append</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="c1"># Create the node for this step in the lineage chain, and attach it to the previous one.</span>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">Node</span><span class="p">(</span>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">scope_name</span><span class="si">}</span><span class="s2">.</span><span class="si">{</span><span class="n">column</span><span class="si">}</span><span class="s2">"</span> <span class="k">if</span> <span class="n">scope_name</span> <span class="k">else</span> <span class="nb">str</span><span class="p">(</span><span class="n">column</span><span class="p">),</span>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span>
|
||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="n">expression</span><span class="o">=</span><span class="n">select</span><span class="p">,</span>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">alias</span> <span class="ow">or</span> <span class="s2">""</span><span class="p">,</span>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="p">)</span>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">if</span> <span class="n">upstream</span><span class="p">:</span>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="n">upstream</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="c1"># Find all columns that went into creating this one to list their lineage nodes.</span>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="nb">set</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)):</span>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">c</span><span class="o">.</span><span class="n">table</span>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="k">return</span> <span class="n">to_node</span><span class="p">(</span><span class="n">column</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="p">)</span>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a><span class="k">class</span> <span class="nc">LineageHTML</span><span class="p">:</span>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a><span class="w"> </span><span class="sd">"""Node to HTML generator using vis.js.</span>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a><span class="sd"> https://visjs.github.io/vis-network/docs/network/</span>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="bp">self</span><span class="p">,</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="n">node</span><span class="p">:</span> <span class="n">Node</span><span class="p">,</span>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="n">imports</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="p">):</span>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">node</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span> <span class="o">=</span> <span class="n">imports</span>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="s2">"height"</span><span class="p">:</span> <span class="s2">"500px"</span><span class="p">,</span>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="s2">"width"</span><span class="p">:</span> <span class="s2">"100%"</span><span class="p">,</span>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="s2">"layout"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="s2">"hierarchical"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="s2">"enabled"</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="s2">"nodeSpacing"</span><span class="p">:</span> <span class="mi">200</span><span class="p">,</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="s2">"sortMethod"</span><span class="p">:</span> <span class="s2">"directed"</span><span class="p">,</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="p">},</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="p">},</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="s2">"interaction"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="s2">"dragNodes"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="s2">"selectable"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="p">},</span>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="s2">"physics"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="s2">"enabled"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="p">},</span>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="s2">"edges"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="s2">"arrows"</span><span class="p">:</span> <span class="s2">"to"</span><span class="p">,</span>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="p">},</span>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="s2">"nodes"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="s2">"font"</span><span class="p">:</span> <span class="s2">"20px monaco"</span><span class="p">,</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="s2">"shape"</span><span class="p">:</span> <span class="s2">"box"</span><span class="p">,</span>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="s2">"widthConstraint"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="s2">"maximum"</span><span class="p">:</span> <span class="mi">300</span><span class="p">,</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="p">},</span>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="p">},</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">,</span>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="p">}</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a>
|
||||
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
|
||||
</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a> <span class="n">label</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">"</span>
|
||||
</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"<pre>SELECT </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2"> FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2"></pre>"</span>
|
||||
</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">1</span>
|
||||
</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a> <span class="n">label</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">source</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
|
||||
</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a> <span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Tag</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">n</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">"<b>"</span><span class="p">,</span> <span class="n">postfix</span><span class="o">=</span><span class="s2">"</b>"</span><span class="p">)</span>
|
||||
</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a> <span class="k">else</span> <span class="n">n</span><span class="p">,</span>
|
||||
</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a> <span class="p">)</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"<pre></span><span class="si">{</span><span class="n">source</span><span class="si">}</span><span class="s2"></pre>"</span>
|
||||
</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
</span><span id="L-219"><a href="#L-219"><span class="linenos">219</span></a>
|
||||
</span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a> <span class="n">node_id</span> <span class="o">=</span> <span class="nb">id</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a>
|
||||
</span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="p">[</span><span class="n">node_id</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="L-223"><a href="#L-223"><span class="linenos">223</span></a> <span class="s2">"id"</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span>
|
||||
</span><span id="L-224"><a href="#L-224"><span class="linenos">224</span></a> <span class="s2">"label"</span><span class="p">:</span> <span class="n">label</span><span class="p">,</span>
|
||||
</span><span id="L-225"><a href="#L-225"><span class="linenos">225</span></a> <span class="s2">"title"</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span>
|
||||
</span><span id="L-226"><a href="#L-226"><span class="linenos">226</span></a> <span class="s2">"group"</span><span class="p">:</span> <span class="n">group</span><span class="p">,</span>
|
||||
</span><span id="L-227"><a href="#L-227"><span class="linenos">227</span></a> <span class="p">}</span>
|
||||
</span><span id="L-228"><a href="#L-228"><span class="linenos">228</span></a>
|
||||
</span><span id="L-229"><a href="#L-229"><span class="linenos">229</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="p">:</span>
|
||||
</span><span id="L-230"><a href="#L-230"><span class="linenos">230</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s2">"from"</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> <span class="s2">"to"</span><span class="p">:</span> <span class="nb">id</span><span class="p">(</span><span class="n">d</span><span class="p">)})</span>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="c1"># The table itself came from a more specific scope. Recurse into that one using the unaliased column name.</span>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="n">to_node</span><span class="p">(</span>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="n">c</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">scope_name</span><span class="o">=</span><span class="n">table</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">node</span><span class="p">,</span> <span class="n">alias</span><span class="o">=</span><span class="n">aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="p">)</span>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="c1"># The source is not a scope - we've reached the end of the line. At this point, if a source is not found</span>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="c1"># it means this column's lineage is unknown. This can happen if the definition of a source used in a query</span>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="c1"># is not passed into the `sources` map.</span>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">source</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">Placeholder</span><span class="p">()</span>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="n">c</span><span class="o">.</span><span class="n">sql</span><span class="p">(),</span> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">source</span><span class="p">))</span>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="k">return</span> <span class="n">to_node</span><span class="p">(</span><span class="n">column</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="p">)</span>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a><span class="k">class</span> <span class="nc">LineageHTML</span><span class="p">:</span>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a><span class="w"> </span><span class="sd">"""Node to HTML generator using vis.js.</span>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a><span class="sd"> https://visjs.github.io/vis-network/docs/network/</span>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="bp">self</span><span class="p">,</span>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="n">node</span><span class="p">:</span> <span class="n">Node</span><span class="p">,</span>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="n">imports</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="p">):</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">node</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span> <span class="o">=</span> <span class="n">imports</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="s2">"height"</span><span class="p">:</span> <span class="s2">"500px"</span><span class="p">,</span>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="s2">"width"</span><span class="p">:</span> <span class="s2">"100%"</span><span class="p">,</span>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="s2">"layout"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="s2">"hierarchical"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="s2">"enabled"</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="s2">"nodeSpacing"</span><span class="p">:</span> <span class="mi">200</span><span class="p">,</span>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="s2">"sortMethod"</span><span class="p">:</span> <span class="s2">"directed"</span><span class="p">,</span>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="p">},</span>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="p">},</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="s2">"interaction"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="s2">"dragNodes"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="s2">"selectable"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="p">},</span>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="s2">"physics"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="s2">"enabled"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="p">},</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="s2">"edges"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="s2">"arrows"</span><span class="p">:</span> <span class="s2">"to"</span><span class="p">,</span>
|
||||
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="p">},</span>
|
||||
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a> <span class="s2">"nodes"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a> <span class="s2">"font"</span><span class="p">:</span> <span class="s2">"20px monaco"</span><span class="p">,</span>
|
||||
</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a> <span class="s2">"shape"</span><span class="p">:</span> <span class="s2">"box"</span><span class="p">,</span>
|
||||
</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a> <span class="s2">"widthConstraint"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a> <span class="s2">"maximum"</span><span class="p">:</span> <span class="mi">300</span><span class="p">,</span>
|
||||
</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a> <span class="p">},</span>
|
||||
</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a> <span class="p">},</span>
|
||||
</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">,</span>
|
||||
</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="p">}</span>
|
||||
</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a>
|
||||
</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a>
|
||||
</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
|
||||
</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a> <span class="n">label</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">"</span>
|
||||
</span><span id="L-219"><a href="#L-219"><span class="linenos">219</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"<pre>SELECT </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2"> FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2"></pre>"</span>
|
||||
</span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">1</span>
|
||||
</span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a> <span class="n">label</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-223"><a href="#L-223"><span class="linenos">223</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">source</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
|
||||
</span><span id="L-224"><a href="#L-224"><span class="linenos">224</span></a> <span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Tag</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">n</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">"<b>"</span><span class="p">,</span> <span class="n">postfix</span><span class="o">=</span><span class="s2">"</b>"</span><span class="p">)</span>
|
||||
</span><span id="L-225"><a href="#L-225"><span class="linenos">225</span></a> <span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-226"><a href="#L-226"><span class="linenos">226</span></a> <span class="k">else</span> <span class="n">n</span><span class="p">,</span>
|
||||
</span><span id="L-227"><a href="#L-227"><span class="linenos">227</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-228"><a href="#L-228"><span class="linenos">228</span></a> <span class="p">)</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-229"><a href="#L-229"><span class="linenos">229</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"<pre></span><span class="si">{</span><span class="n">source</span><span class="si">}</span><span class="s2"></pre>"</span>
|
||||
</span><span id="L-230"><a href="#L-230"><span class="linenos">230</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
</span><span id="L-231"><a href="#L-231"><span class="linenos">231</span></a>
|
||||
</span><span id="L-232"><a href="#L-232"><span class="linenos">232</span></a> <span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-233"><a href="#L-233"><span class="linenos">233</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="o">.</span><span class="n">values</span><span class="p">()))</span>
|
||||
</span><span id="L-234"><a href="#L-234"><span class="linenos">234</span></a> <span class="n">edges</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="p">)</span>
|
||||
</span><span id="L-235"><a href="#L-235"><span class="linenos">235</span></a> <span class="n">options</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">options</span><span class="p">)</span>
|
||||
</span><span id="L-236"><a href="#L-236"><span class="linenos">236</span></a> <span class="n">imports</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a><span class="w"> </span><span class="sd">"""<script type="text/javascript" src="https://unpkg.com/vis-data@latest/peer/umd/vis-data.min.js"></script></span>
|
||||
</span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a><span class="sd"> <script type="text/javascript" src="https://unpkg.com/vis-network@latest/peer/umd/vis-network.min.js"></script></span>
|
||||
</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a><span class="sd"> <link rel="stylesheet" type="text/css" href="https://unpkg.com/vis-network/styles/vis-network.min.css" />"""</span>
|
||||
</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span>
|
||||
</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a> <span class="k">else</span> <span class="s2">""</span>
|
||||
</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a> <span class="p">)</span>
|
||||
</span><span id="L-232"><a href="#L-232"><span class="linenos">232</span></a> <span class="n">node_id</span> <span class="o">=</span> <span class="nb">id</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-233"><a href="#L-233"><span class="linenos">233</span></a>
|
||||
</span><span id="L-234"><a href="#L-234"><span class="linenos">234</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="p">[</span><span class="n">node_id</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="L-235"><a href="#L-235"><span class="linenos">235</span></a> <span class="s2">"id"</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span>
|
||||
</span><span id="L-236"><a href="#L-236"><span class="linenos">236</span></a> <span class="s2">"label"</span><span class="p">:</span> <span class="n">label</span><span class="p">,</span>
|
||||
</span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a> <span class="s2">"title"</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span>
|
||||
</span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a> <span class="s2">"group"</span><span class="p">:</span> <span class="n">group</span><span class="p">,</span>
|
||||
</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a> <span class="p">}</span>
|
||||
</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a>
|
||||
</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="p">:</span>
|
||||
</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s2">"from"</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> <span class="s2">"to"</span><span class="p">:</span> <span class="nb">id</span><span class="p">(</span><span class="n">d</span><span class="p">)})</span>
|
||||
</span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a>
|
||||
</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="k">return</span> <span class="sa">f</span><span class="s2">"""<div></span>
|
||||
</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a><span class="s2"> <div id="sqlglot-lineage"></div></span>
|
||||
</span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a><span class="s2"> </span><span class="si">{</span><span class="n">imports</span><span class="si">}</span>
|
||||
</span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a><span class="s2"> <script type="text/javascript"></span>
|
||||
</span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a><span class="s2"> var nodes = new vis.DataSet(</span><span class="si">{</span><span class="n">nodes</span><span class="si">}</span><span class="s2">)</span>
|
||||
</span><span id="L-249"><a href="#L-249"><span class="linenos">249</span></a><span class="s2"> nodes.forEach(row => row["title"] = new DOMParser().parseFromString(row["title"], "text/html").body.childNodes[0])</span>
|
||||
</span><span id="L-250"><a href="#L-250"><span class="linenos">250</span></a>
|
||||
</span><span id="L-251"><a href="#L-251"><span class="linenos">251</span></a><span class="s2"> new vis.Network(</span>
|
||||
</span><span id="L-252"><a href="#L-252"><span class="linenos">252</span></a><span class="s2"> document.getElementById("sqlglot-lineage"),</span>
|
||||
</span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a><span class="s2"> </span><span class="se">{{</span>
|
||||
</span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a><span class="s2"> nodes: nodes,</span>
|
||||
</span><span id="L-255"><a href="#L-255"><span class="linenos">255</span></a><span class="s2"> edges: new vis.DataSet(</span><span class="si">{</span><span class="n">edges</span><span class="si">}</span><span class="s2">)</span>
|
||||
</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a><span class="s2"> </span><span class="se">}}</span><span class="s2">,</span>
|
||||
</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a><span class="s2"> </span><span class="si">{</span><span class="n">options</span><span class="si">}</span><span class="s2">,</span>
|
||||
</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a><span class="s2"> )</span>
|
||||
</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a><span class="s2"> </script></span>
|
||||
</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a><span class="s2"></div>"""</span>
|
||||
</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a>
|
||||
</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a> <span class="k">def</span> <span class="nf">_repr_html_</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="fm">__str__</span><span class="p">()</span>
|
||||
</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="o">.</span><span class="n">values</span><span class="p">()))</span>
|
||||
</span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a> <span class="n">edges</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="p">)</span>
|
||||
</span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a> <span class="n">options</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">options</span><span class="p">)</span>
|
||||
</span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a> <span class="n">imports</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="L-249"><a href="#L-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""<script type="text/javascript" src="https://unpkg.com/vis-data@latest/peer/umd/vis-data.min.js"></script></span>
|
||||
</span><span id="L-250"><a href="#L-250"><span class="linenos">250</span></a><span class="sd"> <script type="text/javascript" src="https://unpkg.com/vis-network@latest/peer/umd/vis-network.min.js"></script></span>
|
||||
</span><span id="L-251"><a href="#L-251"><span class="linenos">251</span></a><span class="sd"> <link rel="stylesheet" type="text/css" href="https://unpkg.com/vis-network/styles/vis-network.min.css" />"""</span>
|
||||
</span><span id="L-252"><a href="#L-252"><span class="linenos">252</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span>
|
||||
</span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a> <span class="k">else</span> <span class="s2">""</span>
|
||||
</span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a> <span class="p">)</span>
|
||||
</span><span id="L-255"><a href="#L-255"><span class="linenos">255</span></a>
|
||||
</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a> <span class="k">return</span> <span class="sa">f</span><span class="s2">"""<div></span>
|
||||
</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a><span class="s2"> <div id="sqlglot-lineage"></div></span>
|
||||
</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a><span class="s2"> </span><span class="si">{</span><span class="n">imports</span><span class="si">}</span>
|
||||
</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a><span class="s2"> <script type="text/javascript"></span>
|
||||
</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a><span class="s2"> var nodes = new vis.DataSet(</span><span class="si">{</span><span class="n">nodes</span><span class="si">}</span><span class="s2">)</span>
|
||||
</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a><span class="s2"> nodes.forEach(row => row["title"] = new DOMParser().parseFromString(row["title"], "text/html").body.childNodes[0])</span>
|
||||
</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a>
|
||||
</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a><span class="s2"> new vis.Network(</span>
|
||||
</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a><span class="s2"> document.getElementById("sqlglot-lineage"),</span>
|
||||
</span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a><span class="s2"> </span><span class="se">{{</span>
|
||||
</span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a><span class="s2"> nodes: nodes,</span>
|
||||
</span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a><span class="s2"> edges: new vis.DataSet(</span><span class="si">{</span><span class="n">edges</span><span class="si">}</span><span class="s2">)</span>
|
||||
</span><span id="L-268"><a href="#L-268"><span class="linenos">268</span></a><span class="s2"> </span><span class="se">}}</span><span class="s2">,</span>
|
||||
</span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a><span class="s2"> </span><span class="si">{</span><span class="n">options</span><span class="si">}</span><span class="s2">,</span>
|
||||
</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a><span class="s2"> )</span>
|
||||
</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a><span class="s2"> </script></span>
|
||||
</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a><span class="s2"></div>"""</span>
|
||||
</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a>
|
||||
</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a> <span class="k">def</span> <span class="nf">_repr_html_</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="fm">__str__</span><span class="p">()</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -583,7 +595,7 @@
|
|||
</span><span id="lineage-80"><a href="#lineage-80"><span class="linenos"> 80</span></a> <span class="k">raise</span> <span class="n">SqlglotError</span><span class="p">(</span><span class="s2">"Cannot build lineage, sql must be SELECT"</span><span class="p">)</span>
|
||||
</span><span id="lineage-81"><a href="#lineage-81"><span class="linenos"> 81</span></a>
|
||||
</span><span id="lineage-82"><a href="#lineage-82"><span class="linenos"> 82</span></a> <span class="k">def</span> <span class="nf">to_node</span><span class="p">(</span>
|
||||
</span><span id="lineage-83"><a href="#lineage-83"><span class="linenos"> 83</span></a> <span class="n">column_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span>
|
||||
</span><span id="lineage-83"><a href="#lineage-83"><span class="linenos"> 83</span></a> <span class="n">column</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">int</span><span class="p">,</span>
|
||||
</span><span id="lineage-84"><a href="#lineage-84"><span class="linenos"> 84</span></a> <span class="n">scope</span><span class="p">:</span> <span class="n">Scope</span><span class="p">,</span>
|
||||
</span><span id="lineage-85"><a href="#lineage-85"><span class="linenos"> 85</span></a> <span class="n">scope_name</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="lineage-86"><a href="#lineage-86"><span class="linenos"> 86</span></a> <span class="n">upstream</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Node</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
|
|
@ -594,66 +606,78 @@
|
|||
</span><span id="lineage-91"><a href="#lineage-91"><span class="linenos"> 91</span></a> <span class="k">for</span> <span class="n">dt</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span>
|
||||
</span><span id="lineage-92"><a href="#lineage-92"><span class="linenos"> 92</span></a> <span class="k">if</span> <span class="n">dt</span><span class="o">.</span><span class="n">comments</span> <span class="ow">and</span> <span class="n">dt</span><span class="o">.</span><span class="n">comments</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"source: "</span><span class="p">)</span>
|
||||
</span><span id="lineage-93"><a href="#lineage-93"><span class="linenos"> 93</span></a> <span class="p">}</span>
|
||||
</span><span id="lineage-94"><a href="#lineage-94"><span class="linenos"> 94</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Union</span><span class="p">):</span>
|
||||
</span><span id="lineage-95"><a href="#lineage-95"><span class="linenos"> 95</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">union_scopes</span><span class="p">:</span>
|
||||
</span><span id="lineage-96"><a href="#lineage-96"><span class="linenos"> 96</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">to_node</span><span class="p">(</span>
|
||||
</span><span id="lineage-97"><a href="#lineage-97"><span class="linenos"> 97</span></a> <span class="n">column_name</span><span class="p">,</span>
|
||||
</span><span id="lineage-98"><a href="#lineage-98"><span class="linenos"> 98</span></a> <span class="n">scope</span><span class="o">=</span><span class="n">scope</span><span class="p">,</span>
|
||||
</span><span id="lineage-99"><a href="#lineage-99"><span class="linenos"> 99</span></a> <span class="n">scope_name</span><span class="o">=</span><span class="n">scope_name</span><span class="p">,</span>
|
||||
</span><span id="lineage-100"><a href="#lineage-100"><span class="linenos">100</span></a> <span class="n">upstream</span><span class="o">=</span><span class="n">upstream</span><span class="p">,</span>
|
||||
</span><span id="lineage-101"><a href="#lineage-101"><span class="linenos">101</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">scope_name</span><span class="p">),</span>
|
||||
</span><span id="lineage-102"><a href="#lineage-102"><span class="linenos">102</span></a> <span class="p">)</span>
|
||||
</span><span id="lineage-103"><a href="#lineage-103"><span class="linenos">103</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
</span><span id="lineage-104"><a href="#lineage-104"><span class="linenos">104</span></a>
|
||||
</span><span id="lineage-105"><a href="#lineage-105"><span class="linenos">105</span></a> <span class="c1"># Find the specific select clause that is the source of the column we want.</span>
|
||||
</span><span id="lineage-106"><a href="#lineage-106"><span class="linenos">106</span></a> <span class="c1"># This can either be a specific, named select or a generic `*` clause.</span>
|
||||
</span><span id="lineage-107"><a href="#lineage-107"><span class="linenos">107</span></a> <span class="n">select</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span>
|
||||
</span><span id="lineage-108"><a href="#lineage-108"><span class="linenos">108</span></a> <span class="p">(</span><span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">column_name</span><span class="p">),</span>
|
||||
</span><span id="lineage-109"><a href="#lineage-109"><span class="linenos">109</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Star</span><span class="p">()</span> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">is_star</span> <span class="k">else</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="lineage-110"><a href="#lineage-110"><span class="linenos">110</span></a> <span class="p">)</span>
|
||||
</span><span id="lineage-94"><a href="#lineage-94"><span class="linenos"> 94</span></a>
|
||||
</span><span id="lineage-95"><a href="#lineage-95"><span class="linenos"> 95</span></a> <span class="c1"># Find the specific select clause that is the source of the column we want.</span>
|
||||
</span><span id="lineage-96"><a href="#lineage-96"><span class="linenos"> 96</span></a> <span class="c1"># This can either be a specific, named select or a generic `*` clause.</span>
|
||||
</span><span id="lineage-97"><a href="#lineage-97"><span class="linenos"> 97</span></a> <span class="n">select</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="lineage-98"><a href="#lineage-98"><span class="linenos"> 98</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">[</span><span class="n">column</span><span class="p">]</span>
|
||||
</span><span id="lineage-99"><a href="#lineage-99"><span class="linenos"> 99</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">int</span><span class="p">)</span>
|
||||
</span><span id="lineage-100"><a href="#lineage-100"><span class="linenos">100</span></a> <span class="k">else</span> <span class="nb">next</span><span class="p">(</span>
|
||||
</span><span id="lineage-101"><a href="#lineage-101"><span class="linenos">101</span></a> <span class="p">(</span><span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">column</span><span class="p">),</span>
|
||||
</span><span id="lineage-102"><a href="#lineage-102"><span class="linenos">102</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Star</span><span class="p">()</span> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">is_star</span> <span class="k">else</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="lineage-103"><a href="#lineage-103"><span class="linenos">103</span></a> <span class="p">)</span>
|
||||
</span><span id="lineage-104"><a href="#lineage-104"><span class="linenos">104</span></a> <span class="p">)</span>
|
||||
</span><span id="lineage-105"><a href="#lineage-105"><span class="linenos">105</span></a>
|
||||
</span><span id="lineage-106"><a href="#lineage-106"><span class="linenos">106</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">select</span><span class="p">:</span>
|
||||
</span><span id="lineage-107"><a href="#lineage-107"><span class="linenos">107</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Could not find </span><span class="si">{</span><span class="n">column</span><span class="si">}</span><span class="s2"> in </span><span class="si">{</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="lineage-108"><a href="#lineage-108"><span class="linenos">108</span></a>
|
||||
</span><span id="lineage-109"><a href="#lineage-109"><span class="linenos">109</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Union</span><span class="p">):</span>
|
||||
</span><span id="lineage-110"><a href="#lineage-110"><span class="linenos">110</span></a> <span class="n">upstream</span> <span class="o">=</span> <span class="n">upstream</span> <span class="ow">or</span> <span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s2">"UNION"</span><span class="p">,</span> <span class="n">source</span><span class="o">=</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">select</span><span class="p">)</span>
|
||||
</span><span id="lineage-111"><a href="#lineage-111"><span class="linenos">111</span></a>
|
||||
</span><span id="lineage-112"><a href="#lineage-112"><span class="linenos">112</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">select</span><span class="p">:</span>
|
||||
</span><span id="lineage-113"><a href="#lineage-113"><span class="linenos">113</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Could not find </span><span class="si">{</span><span class="n">column_name</span><span class="si">}</span><span class="s2"> in </span><span class="si">{</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="lineage-114"><a href="#lineage-114"><span class="linenos">114</span></a>
|
||||
</span><span id="lineage-115"><a href="#lineage-115"><span class="linenos">115</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="lineage-116"><a href="#lineage-116"><span class="linenos">116</span></a> <span class="c1"># For better ergonomics in our node labels, replace the full select with</span>
|
||||
</span><span id="lineage-117"><a href="#lineage-117"><span class="linenos">117</span></a> <span class="c1"># a version that has only the column we care about.</span>
|
||||
</span><span id="lineage-118"><a href="#lineage-118"><span class="linenos">118</span></a> <span class="c1"># "x", SELECT x, y FROM foo</span>
|
||||
</span><span id="lineage-119"><a href="#lineage-119"><span class="linenos">119</span></a> <span class="c1"># => "x", SELECT x FROM foo</span>
|
||||
</span><span id="lineage-120"><a href="#lineage-120"><span class="linenos">120</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">append</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
|
||||
</span><span id="lineage-121"><a href="#lineage-121"><span class="linenos">121</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="lineage-122"><a href="#lineage-122"><span class="linenos">122</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="lineage-123"><a href="#lineage-123"><span class="linenos">123</span></a>
|
||||
</span><span id="lineage-124"><a href="#lineage-124"><span class="linenos">124</span></a> <span class="c1"># Create the node for this step in the lineage chain, and attach it to the previous one.</span>
|
||||
</span><span id="lineage-125"><a href="#lineage-125"><span class="linenos">125</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">Node</span><span class="p">(</span>
|
||||
</span><span id="lineage-126"><a href="#lineage-126"><span class="linenos">126</span></a> <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">scope_name</span><span class="si">}</span><span class="s2">.</span><span class="si">{</span><span class="n">column_name</span><span class="si">}</span><span class="s2">"</span> <span class="k">if</span> <span class="n">scope_name</span> <span class="k">else</span> <span class="n">column_name</span><span class="p">,</span>
|
||||
</span><span id="lineage-127"><a href="#lineage-127"><span class="linenos">127</span></a> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span>
|
||||
</span><span id="lineage-128"><a href="#lineage-128"><span class="linenos">128</span></a> <span class="n">expression</span><span class="o">=</span><span class="n">select</span><span class="p">,</span>
|
||||
</span><span id="lineage-129"><a href="#lineage-129"><span class="linenos">129</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">alias</span> <span class="ow">or</span> <span class="s2">""</span><span class="p">,</span>
|
||||
</span><span id="lineage-130"><a href="#lineage-130"><span class="linenos">130</span></a> <span class="p">)</span>
|
||||
</span><span id="lineage-131"><a href="#lineage-131"><span class="linenos">131</span></a> <span class="k">if</span> <span class="n">upstream</span><span class="p">:</span>
|
||||
</span><span id="lineage-132"><a href="#lineage-132"><span class="linenos">132</span></a> <span class="n">upstream</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="lineage-133"><a href="#lineage-133"><span class="linenos">133</span></a>
|
||||
</span><span id="lineage-134"><a href="#lineage-134"><span class="linenos">134</span></a> <span class="c1"># Find all columns that went into creating this one to list their lineage nodes.</span>
|
||||
</span><span id="lineage-135"><a href="#lineage-135"><span class="linenos">135</span></a> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="nb">set</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)):</span>
|
||||
</span><span id="lineage-136"><a href="#lineage-136"><span class="linenos">136</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">c</span><span class="o">.</span><span class="n">table</span>
|
||||
</span><span id="lineage-137"><a href="#lineage-137"><span class="linenos">137</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
|
||||
</span><span id="lineage-138"><a href="#lineage-138"><span class="linenos">138</span></a>
|
||||
</span><span id="lineage-139"><a href="#lineage-139"><span class="linenos">139</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span>
|
||||
</span><span id="lineage-140"><a href="#lineage-140"><span class="linenos">140</span></a> <span class="c1"># The table itself came from a more specific scope. Recurse into that one using the unaliased column name.</span>
|
||||
</span><span id="lineage-141"><a href="#lineage-141"><span class="linenos">141</span></a> <span class="n">to_node</span><span class="p">(</span>
|
||||
</span><span id="lineage-142"><a href="#lineage-142"><span class="linenos">142</span></a> <span class="n">c</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">scope_name</span><span class="o">=</span><span class="n">table</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">node</span><span class="p">,</span> <span class="n">alias</span><span class="o">=</span><span class="n">aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
|
||||
</span><span id="lineage-143"><a href="#lineage-143"><span class="linenos">143</span></a> <span class="p">)</span>
|
||||
</span><span id="lineage-144"><a href="#lineage-144"><span class="linenos">144</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="lineage-145"><a href="#lineage-145"><span class="linenos">145</span></a> <span class="c1"># The source is not a scope - we've reached the end of the line. At this point, if a source is not found</span>
|
||||
</span><span id="lineage-146"><a href="#lineage-146"><span class="linenos">146</span></a> <span class="c1"># it means this column's lineage is unknown. This can happen if the definition of a source used in a query</span>
|
||||
</span><span id="lineage-147"><a href="#lineage-147"><span class="linenos">147</span></a> <span class="c1"># is not passed into the `sources` map.</span>
|
||||
</span><span id="lineage-148"><a href="#lineage-148"><span class="linenos">148</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">source</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">Placeholder</span><span class="p">()</span>
|
||||
</span><span id="lineage-149"><a href="#lineage-149"><span class="linenos">149</span></a> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="n">c</span><span class="o">.</span><span class="n">sql</span><span class="p">(),</span> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">source</span><span class="p">))</span>
|
||||
</span><span id="lineage-112"><a href="#lineage-112"><span class="linenos">112</span></a> <span class="n">index</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="lineage-113"><a href="#lineage-113"><span class="linenos">113</span></a> <span class="n">column</span>
|
||||
</span><span id="lineage-114"><a href="#lineage-114"><span class="linenos">114</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">int</span><span class="p">)</span>
|
||||
</span><span id="lineage-115"><a href="#lineage-115"><span class="linenos">115</span></a> <span class="k">else</span> <span class="nb">next</span><span class="p">(</span>
|
||||
</span><span id="lineage-116"><a href="#lineage-116"><span class="linenos">116</span></a> <span class="n">i</span>
|
||||
</span><span id="lineage-117"><a href="#lineage-117"><span class="linenos">117</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">select</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span>
|
||||
</span><span id="lineage-118"><a href="#lineage-118"><span class="linenos">118</span></a> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">column</span>
|
||||
</span><span id="lineage-119"><a href="#lineage-119"><span class="linenos">119</span></a> <span class="p">)</span>
|
||||
</span><span id="lineage-120"><a href="#lineage-120"><span class="linenos">120</span></a> <span class="p">)</span>
|
||||
</span><span id="lineage-121"><a href="#lineage-121"><span class="linenos">121</span></a>
|
||||
</span><span id="lineage-122"><a href="#lineage-122"><span class="linenos">122</span></a> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">union_scopes</span><span class="p">:</span>
|
||||
</span><span id="lineage-123"><a href="#lineage-123"><span class="linenos">123</span></a> <span class="n">to_node</span><span class="p">(</span><span class="n">index</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">s</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">upstream</span><span class="p">)</span>
|
||||
</span><span id="lineage-124"><a href="#lineage-124"><span class="linenos">124</span></a>
|
||||
</span><span id="lineage-125"><a href="#lineage-125"><span class="linenos">125</span></a> <span class="k">return</span> <span class="n">upstream</span>
|
||||
</span><span id="lineage-126"><a href="#lineage-126"><span class="linenos">126</span></a>
|
||||
</span><span id="lineage-127"><a href="#lineage-127"><span class="linenos">127</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="lineage-128"><a href="#lineage-128"><span class="linenos">128</span></a> <span class="c1"># For better ergonomics in our node labels, replace the full select with</span>
|
||||
</span><span id="lineage-129"><a href="#lineage-129"><span class="linenos">129</span></a> <span class="c1"># a version that has only the column we care about.</span>
|
||||
</span><span id="lineage-130"><a href="#lineage-130"><span class="linenos">130</span></a> <span class="c1"># "x", SELECT x, y FROM foo</span>
|
||||
</span><span id="lineage-131"><a href="#lineage-131"><span class="linenos">131</span></a> <span class="c1"># => "x", SELECT x FROM foo</span>
|
||||
</span><span id="lineage-132"><a href="#lineage-132"><span class="linenos">132</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">append</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
|
||||
</span><span id="lineage-133"><a href="#lineage-133"><span class="linenos">133</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="lineage-134"><a href="#lineage-134"><span class="linenos">134</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="lineage-135"><a href="#lineage-135"><span class="linenos">135</span></a>
|
||||
</span><span id="lineage-136"><a href="#lineage-136"><span class="linenos">136</span></a> <span class="c1"># Create the node for this step in the lineage chain, and attach it to the previous one.</span>
|
||||
</span><span id="lineage-137"><a href="#lineage-137"><span class="linenos">137</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">Node</span><span class="p">(</span>
|
||||
</span><span id="lineage-138"><a href="#lineage-138"><span class="linenos">138</span></a> <span class="n">name</span><span class="o">=</span><span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">scope_name</span><span class="si">}</span><span class="s2">.</span><span class="si">{</span><span class="n">column</span><span class="si">}</span><span class="s2">"</span> <span class="k">if</span> <span class="n">scope_name</span> <span class="k">else</span> <span class="nb">str</span><span class="p">(</span><span class="n">column</span><span class="p">),</span>
|
||||
</span><span id="lineage-139"><a href="#lineage-139"><span class="linenos">139</span></a> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span>
|
||||
</span><span id="lineage-140"><a href="#lineage-140"><span class="linenos">140</span></a> <span class="n">expression</span><span class="o">=</span><span class="n">select</span><span class="p">,</span>
|
||||
</span><span id="lineage-141"><a href="#lineage-141"><span class="linenos">141</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">alias</span> <span class="ow">or</span> <span class="s2">""</span><span class="p">,</span>
|
||||
</span><span id="lineage-142"><a href="#lineage-142"><span class="linenos">142</span></a> <span class="p">)</span>
|
||||
</span><span id="lineage-143"><a href="#lineage-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="n">upstream</span><span class="p">:</span>
|
||||
</span><span id="lineage-144"><a href="#lineage-144"><span class="linenos">144</span></a> <span class="n">upstream</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="lineage-145"><a href="#lineage-145"><span class="linenos">145</span></a>
|
||||
</span><span id="lineage-146"><a href="#lineage-146"><span class="linenos">146</span></a> <span class="c1"># Find all columns that went into creating this one to list their lineage nodes.</span>
|
||||
</span><span id="lineage-147"><a href="#lineage-147"><span class="linenos">147</span></a> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="nb">set</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)):</span>
|
||||
</span><span id="lineage-148"><a href="#lineage-148"><span class="linenos">148</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">c</span><span class="o">.</span><span class="n">table</span>
|
||||
</span><span id="lineage-149"><a href="#lineage-149"><span class="linenos">149</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
|
||||
</span><span id="lineage-150"><a href="#lineage-150"><span class="linenos">150</span></a>
|
||||
</span><span id="lineage-151"><a href="#lineage-151"><span class="linenos">151</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
</span><span id="lineage-152"><a href="#lineage-152"><span class="linenos">152</span></a>
|
||||
</span><span id="lineage-153"><a href="#lineage-153"><span class="linenos">153</span></a> <span class="k">return</span> <span class="n">to_node</span><span class="p">(</span><span class="n">column</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="p">)</span>
|
||||
</span><span id="lineage-151"><a href="#lineage-151"><span class="linenos">151</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span>
|
||||
</span><span id="lineage-152"><a href="#lineage-152"><span class="linenos">152</span></a> <span class="c1"># The table itself came from a more specific scope. Recurse into that one using the unaliased column name.</span>
|
||||
</span><span id="lineage-153"><a href="#lineage-153"><span class="linenos">153</span></a> <span class="n">to_node</span><span class="p">(</span>
|
||||
</span><span id="lineage-154"><a href="#lineage-154"><span class="linenos">154</span></a> <span class="n">c</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">scope_name</span><span class="o">=</span><span class="n">table</span><span class="p">,</span> <span class="n">upstream</span><span class="o">=</span><span class="n">node</span><span class="p">,</span> <span class="n">alias</span><span class="o">=</span><span class="n">aliases</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
|
||||
</span><span id="lineage-155"><a href="#lineage-155"><span class="linenos">155</span></a> <span class="p">)</span>
|
||||
</span><span id="lineage-156"><a href="#lineage-156"><span class="linenos">156</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="lineage-157"><a href="#lineage-157"><span class="linenos">157</span></a> <span class="c1"># The source is not a scope - we've reached the end of the line. At this point, if a source is not found</span>
|
||||
</span><span id="lineage-158"><a href="#lineage-158"><span class="linenos">158</span></a> <span class="c1"># it means this column's lineage is unknown. This can happen if the definition of a source used in a query</span>
|
||||
</span><span id="lineage-159"><a href="#lineage-159"><span class="linenos">159</span></a> <span class="c1"># is not passed into the `sources` map.</span>
|
||||
</span><span id="lineage-160"><a href="#lineage-160"><span class="linenos">160</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">source</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">Placeholder</span><span class="p">()</span>
|
||||
</span><span id="lineage-161"><a href="#lineage-161"><span class="linenos">161</span></a> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Node</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="n">c</span><span class="o">.</span><span class="n">sql</span><span class="p">(),</span> <span class="n">source</span><span class="o">=</span><span class="n">source</span><span class="p">,</span> <span class="n">expression</span><span class="o">=</span><span class="n">source</span><span class="p">))</span>
|
||||
</span><span id="lineage-162"><a href="#lineage-162"><span class="linenos">162</span></a>
|
||||
</span><span id="lineage-163"><a href="#lineage-163"><span class="linenos">163</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
</span><span id="lineage-164"><a href="#lineage-164"><span class="linenos">164</span></a>
|
||||
</span><span id="lineage-165"><a href="#lineage-165"><span class="linenos">165</span></a> <span class="k">return</span> <span class="n">to_node</span><span class="p">(</span><span class="n">column</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">scope</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -690,115 +714,115 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#LineageHTML"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="LineageHTML-156"><a href="#LineageHTML-156"><span class="linenos">156</span></a><span class="k">class</span> <span class="nc">LineageHTML</span><span class="p">:</span>
|
||||
</span><span id="LineageHTML-157"><a href="#LineageHTML-157"><span class="linenos">157</span></a><span class="w"> </span><span class="sd">"""Node to HTML generator using vis.js.</span>
|
||||
</span><span id="LineageHTML-158"><a href="#LineageHTML-158"><span class="linenos">158</span></a>
|
||||
</span><span id="LineageHTML-159"><a href="#LineageHTML-159"><span class="linenos">159</span></a><span class="sd"> https://visjs.github.io/vis-network/docs/network/</span>
|
||||
</span><span id="LineageHTML-160"><a href="#LineageHTML-160"><span class="linenos">160</span></a><span class="sd"> """</span>
|
||||
</span><span id="LineageHTML-161"><a href="#LineageHTML-161"><span class="linenos">161</span></a>
|
||||
</span><span id="LineageHTML-162"><a href="#LineageHTML-162"><span class="linenos">162</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span>
|
||||
</span><span id="LineageHTML-163"><a href="#LineageHTML-163"><span class="linenos">163</span></a> <span class="bp">self</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-164"><a href="#LineageHTML-164"><span class="linenos">164</span></a> <span class="n">node</span><span class="p">:</span> <span class="n">Node</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-165"><a href="#LineageHTML-165"><span class="linenos">165</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-166"><a href="#LineageHTML-166"><span class="linenos">166</span></a> <span class="n">imports</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-167"><a href="#LineageHTML-167"><span class="linenos">167</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-168"><a href="#LineageHTML-168"><span class="linenos">168</span></a> <span class="p">):</span>
|
||||
</span><span id="LineageHTML-169"><a href="#LineageHTML-169"><span class="linenos">169</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">node</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="LineageHTML-170"><a href="#LineageHTML-170"><span class="linenos">170</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span> <span class="o">=</span> <span class="n">imports</span>
|
||||
</span><span id="LineageHTML-171"><a href="#LineageHTML-171"><span class="linenos">171</span></a>
|
||||
</span><span id="LineageHTML-172"><a href="#LineageHTML-172"><span class="linenos">172</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-173"><a href="#LineageHTML-173"><span class="linenos">173</span></a> <span class="s2">"height"</span><span class="p">:</span> <span class="s2">"500px"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-174"><a href="#LineageHTML-174"><span class="linenos">174</span></a> <span class="s2">"width"</span><span class="p">:</span> <span class="s2">"100%"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-175"><a href="#LineageHTML-175"><span class="linenos">175</span></a> <span class="s2">"layout"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-176"><a href="#LineageHTML-176"><span class="linenos">176</span></a> <span class="s2">"hierarchical"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-177"><a href="#LineageHTML-177"><span class="linenos">177</span></a> <span class="s2">"enabled"</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-178"><a href="#LineageHTML-178"><span class="linenos">178</span></a> <span class="s2">"nodeSpacing"</span><span class="p">:</span> <span class="mi">200</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-179"><a href="#LineageHTML-179"><span class="linenos">179</span></a> <span class="s2">"sortMethod"</span><span class="p">:</span> <span class="s2">"directed"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-180"><a href="#LineageHTML-180"><span class="linenos">180</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML-181"><a href="#LineageHTML-181"><span class="linenos">181</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML-182"><a href="#LineageHTML-182"><span class="linenos">182</span></a> <span class="s2">"interaction"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-183"><a href="#LineageHTML-183"><span class="linenos">183</span></a> <span class="s2">"dragNodes"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-184"><a href="#LineageHTML-184"><span class="linenos">184</span></a> <span class="s2">"selectable"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-185"><a href="#LineageHTML-185"><span class="linenos">185</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML-186"><a href="#LineageHTML-186"><span class="linenos">186</span></a> <span class="s2">"physics"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-187"><a href="#LineageHTML-187"><span class="linenos">187</span></a> <span class="s2">"enabled"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-188"><a href="#LineageHTML-188"><span class="linenos">188</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML-189"><a href="#LineageHTML-189"><span class="linenos">189</span></a> <span class="s2">"edges"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-190"><a href="#LineageHTML-190"><span class="linenos">190</span></a> <span class="s2">"arrows"</span><span class="p">:</span> <span class="s2">"to"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-191"><a href="#LineageHTML-191"><span class="linenos">191</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML-192"><a href="#LineageHTML-192"><span class="linenos">192</span></a> <span class="s2">"nodes"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-193"><a href="#LineageHTML-193"><span class="linenos">193</span></a> <span class="s2">"font"</span><span class="p">:</span> <span class="s2">"20px monaco"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-194"><a href="#LineageHTML-194"><span class="linenos">194</span></a> <span class="s2">"shape"</span><span class="p">:</span> <span class="s2">"box"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-195"><a href="#LineageHTML-195"><span class="linenos">195</span></a> <span class="s2">"widthConstraint"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-196"><a href="#LineageHTML-196"><span class="linenos">196</span></a> <span class="s2">"maximum"</span><span class="p">:</span> <span class="mi">300</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-197"><a href="#LineageHTML-197"><span class="linenos">197</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML-198"><a href="#LineageHTML-198"><span class="linenos">198</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML-199"><a href="#LineageHTML-199"><span class="linenos">199</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-200"><a href="#LineageHTML-200"><span class="linenos">200</span></a> <span class="p">}</span>
|
||||
</span><span id="LineageHTML-201"><a href="#LineageHTML-201"><span class="linenos">201</span></a>
|
||||
</span><span id="LineageHTML-202"><a href="#LineageHTML-202"><span class="linenos">202</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="LineageHTML-203"><a href="#LineageHTML-203"><span class="linenos">203</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="LineageHTML-204"><a href="#LineageHTML-204"><span class="linenos">204</span></a>
|
||||
</span><span id="LineageHTML-205"><a href="#LineageHTML-205"><span class="linenos">205</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
|
||||
</span><span id="LineageHTML-206"><a href="#LineageHTML-206"><span class="linenos">206</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="LineageHTML-207"><a href="#LineageHTML-207"><span class="linenos">207</span></a> <span class="n">label</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">"</span>
|
||||
</span><span id="LineageHTML-208"><a href="#LineageHTML-208"><span class="linenos">208</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"<pre>SELECT </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2"> FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2"></pre>"</span>
|
||||
</span><span id="LineageHTML-209"><a href="#LineageHTML-209"><span class="linenos">209</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">1</span>
|
||||
</span><span id="LineageHTML-210"><a href="#LineageHTML-210"><span class="linenos">210</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="LineageHTML-211"><a href="#LineageHTML-211"><span class="linenos">211</span></a> <span class="n">label</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML-212"><a href="#LineageHTML-212"><span class="linenos">212</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">source</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
|
||||
</span><span id="LineageHTML-213"><a href="#LineageHTML-213"><span class="linenos">213</span></a> <span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Tag</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">n</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">"<b>"</span><span class="p">,</span> <span class="n">postfix</span><span class="o">=</span><span class="s2">"</b>"</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML-214"><a href="#LineageHTML-214"><span class="linenos">214</span></a> <span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="LineageHTML-215"><a href="#LineageHTML-215"><span class="linenos">215</span></a> <span class="k">else</span> <span class="n">n</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-216"><a href="#LineageHTML-216"><span class="linenos">216</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-217"><a href="#LineageHTML-217"><span class="linenos">217</span></a> <span class="p">)</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML-218"><a href="#LineageHTML-218"><span class="linenos">218</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"<pre></span><span class="si">{</span><span class="n">source</span><span class="si">}</span><span class="s2"></pre>"</span>
|
||||
</span><span id="LineageHTML-219"><a href="#LineageHTML-219"><span class="linenos">219</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
</span><span id="LineageHTML-220"><a href="#LineageHTML-220"><span class="linenos">220</span></a>
|
||||
</span><span id="LineageHTML-221"><a href="#LineageHTML-221"><span class="linenos">221</span></a> <span class="n">node_id</span> <span class="o">=</span> <span class="nb">id</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML-222"><a href="#LineageHTML-222"><span class="linenos">222</span></a>
|
||||
</span><span id="LineageHTML-223"><a href="#LineageHTML-223"><span class="linenos">223</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="p">[</span><span class="n">node_id</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-224"><a href="#LineageHTML-224"><span class="linenos">224</span></a> <span class="s2">"id"</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-225"><a href="#LineageHTML-225"><span class="linenos">225</span></a> <span class="s2">"label"</span><span class="p">:</span> <span class="n">label</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-226"><a href="#LineageHTML-226"><span class="linenos">226</span></a> <span class="s2">"title"</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-227"><a href="#LineageHTML-227"><span class="linenos">227</span></a> <span class="s2">"group"</span><span class="p">:</span> <span class="n">group</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-228"><a href="#LineageHTML-228"><span class="linenos">228</span></a> <span class="p">}</span>
|
||||
</span><span id="LineageHTML-229"><a href="#LineageHTML-229"><span class="linenos">229</span></a>
|
||||
</span><span id="LineageHTML-230"><a href="#LineageHTML-230"><span class="linenos">230</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="p">:</span>
|
||||
</span><span id="LineageHTML-231"><a href="#LineageHTML-231"><span class="linenos">231</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s2">"from"</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> <span class="s2">"to"</span><span class="p">:</span> <span class="nb">id</span><span class="p">(</span><span class="n">d</span><span class="p">)})</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="LineageHTML-168"><a href="#LineageHTML-168"><span class="linenos">168</span></a><span class="k">class</span> <span class="nc">LineageHTML</span><span class="p">:</span>
|
||||
</span><span id="LineageHTML-169"><a href="#LineageHTML-169"><span class="linenos">169</span></a><span class="w"> </span><span class="sd">"""Node to HTML generator using vis.js.</span>
|
||||
</span><span id="LineageHTML-170"><a href="#LineageHTML-170"><span class="linenos">170</span></a>
|
||||
</span><span id="LineageHTML-171"><a href="#LineageHTML-171"><span class="linenos">171</span></a><span class="sd"> https://visjs.github.io/vis-network/docs/network/</span>
|
||||
</span><span id="LineageHTML-172"><a href="#LineageHTML-172"><span class="linenos">172</span></a><span class="sd"> """</span>
|
||||
</span><span id="LineageHTML-173"><a href="#LineageHTML-173"><span class="linenos">173</span></a>
|
||||
</span><span id="LineageHTML-174"><a href="#LineageHTML-174"><span class="linenos">174</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span>
|
||||
</span><span id="LineageHTML-175"><a href="#LineageHTML-175"><span class="linenos">175</span></a> <span class="bp">self</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-176"><a href="#LineageHTML-176"><span class="linenos">176</span></a> <span class="n">node</span><span class="p">:</span> <span class="n">Node</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-177"><a href="#LineageHTML-177"><span class="linenos">177</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-178"><a href="#LineageHTML-178"><span class="linenos">178</span></a> <span class="n">imports</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-179"><a href="#LineageHTML-179"><span class="linenos">179</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-180"><a href="#LineageHTML-180"><span class="linenos">180</span></a> <span class="p">):</span>
|
||||
</span><span id="LineageHTML-181"><a href="#LineageHTML-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">node</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="LineageHTML-182"><a href="#LineageHTML-182"><span class="linenos">182</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span> <span class="o">=</span> <span class="n">imports</span>
|
||||
</span><span id="LineageHTML-183"><a href="#LineageHTML-183"><span class="linenos">183</span></a>
|
||||
</span><span id="LineageHTML-184"><a href="#LineageHTML-184"><span class="linenos">184</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-185"><a href="#LineageHTML-185"><span class="linenos">185</span></a> <span class="s2">"height"</span><span class="p">:</span> <span class="s2">"500px"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-186"><a href="#LineageHTML-186"><span class="linenos">186</span></a> <span class="s2">"width"</span><span class="p">:</span> <span class="s2">"100%"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-187"><a href="#LineageHTML-187"><span class="linenos">187</span></a> <span class="s2">"layout"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-188"><a href="#LineageHTML-188"><span class="linenos">188</span></a> <span class="s2">"hierarchical"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-189"><a href="#LineageHTML-189"><span class="linenos">189</span></a> <span class="s2">"enabled"</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-190"><a href="#LineageHTML-190"><span class="linenos">190</span></a> <span class="s2">"nodeSpacing"</span><span class="p">:</span> <span class="mi">200</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-191"><a href="#LineageHTML-191"><span class="linenos">191</span></a> <span class="s2">"sortMethod"</span><span class="p">:</span> <span class="s2">"directed"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-192"><a href="#LineageHTML-192"><span class="linenos">192</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML-193"><a href="#LineageHTML-193"><span class="linenos">193</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML-194"><a href="#LineageHTML-194"><span class="linenos">194</span></a> <span class="s2">"interaction"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-195"><a href="#LineageHTML-195"><span class="linenos">195</span></a> <span class="s2">"dragNodes"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-196"><a href="#LineageHTML-196"><span class="linenos">196</span></a> <span class="s2">"selectable"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-197"><a href="#LineageHTML-197"><span class="linenos">197</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML-198"><a href="#LineageHTML-198"><span class="linenos">198</span></a> <span class="s2">"physics"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-199"><a href="#LineageHTML-199"><span class="linenos">199</span></a> <span class="s2">"enabled"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-200"><a href="#LineageHTML-200"><span class="linenos">200</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML-201"><a href="#LineageHTML-201"><span class="linenos">201</span></a> <span class="s2">"edges"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-202"><a href="#LineageHTML-202"><span class="linenos">202</span></a> <span class="s2">"arrows"</span><span class="p">:</span> <span class="s2">"to"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-203"><a href="#LineageHTML-203"><span class="linenos">203</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML-204"><a href="#LineageHTML-204"><span class="linenos">204</span></a> <span class="s2">"nodes"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-205"><a href="#LineageHTML-205"><span class="linenos">205</span></a> <span class="s2">"font"</span><span class="p">:</span> <span class="s2">"20px monaco"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-206"><a href="#LineageHTML-206"><span class="linenos">206</span></a> <span class="s2">"shape"</span><span class="p">:</span> <span class="s2">"box"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-207"><a href="#LineageHTML-207"><span class="linenos">207</span></a> <span class="s2">"widthConstraint"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-208"><a href="#LineageHTML-208"><span class="linenos">208</span></a> <span class="s2">"maximum"</span><span class="p">:</span> <span class="mi">300</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-209"><a href="#LineageHTML-209"><span class="linenos">209</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML-210"><a href="#LineageHTML-210"><span class="linenos">210</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML-211"><a href="#LineageHTML-211"><span class="linenos">211</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-212"><a href="#LineageHTML-212"><span class="linenos">212</span></a> <span class="p">}</span>
|
||||
</span><span id="LineageHTML-213"><a href="#LineageHTML-213"><span class="linenos">213</span></a>
|
||||
</span><span id="LineageHTML-214"><a href="#LineageHTML-214"><span class="linenos">214</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="LineageHTML-215"><a href="#LineageHTML-215"><span class="linenos">215</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="LineageHTML-216"><a href="#LineageHTML-216"><span class="linenos">216</span></a>
|
||||
</span><span id="LineageHTML-217"><a href="#LineageHTML-217"><span class="linenos">217</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
|
||||
</span><span id="LineageHTML-218"><a href="#LineageHTML-218"><span class="linenos">218</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="LineageHTML-219"><a href="#LineageHTML-219"><span class="linenos">219</span></a> <span class="n">label</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">"</span>
|
||||
</span><span id="LineageHTML-220"><a href="#LineageHTML-220"><span class="linenos">220</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"<pre>SELECT </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2"> FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2"></pre>"</span>
|
||||
</span><span id="LineageHTML-221"><a href="#LineageHTML-221"><span class="linenos">221</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">1</span>
|
||||
</span><span id="LineageHTML-222"><a href="#LineageHTML-222"><span class="linenos">222</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="LineageHTML-223"><a href="#LineageHTML-223"><span class="linenos">223</span></a> <span class="n">label</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML-224"><a href="#LineageHTML-224"><span class="linenos">224</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">source</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
|
||||
</span><span id="LineageHTML-225"><a href="#LineageHTML-225"><span class="linenos">225</span></a> <span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Tag</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">n</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">"<b>"</span><span class="p">,</span> <span class="n">postfix</span><span class="o">=</span><span class="s2">"</b>"</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML-226"><a href="#LineageHTML-226"><span class="linenos">226</span></a> <span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="LineageHTML-227"><a href="#LineageHTML-227"><span class="linenos">227</span></a> <span class="k">else</span> <span class="n">n</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-228"><a href="#LineageHTML-228"><span class="linenos">228</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-229"><a href="#LineageHTML-229"><span class="linenos">229</span></a> <span class="p">)</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML-230"><a href="#LineageHTML-230"><span class="linenos">230</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"<pre></span><span class="si">{</span><span class="n">source</span><span class="si">}</span><span class="s2"></pre>"</span>
|
||||
</span><span id="LineageHTML-231"><a href="#LineageHTML-231"><span class="linenos">231</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
</span><span id="LineageHTML-232"><a href="#LineageHTML-232"><span class="linenos">232</span></a>
|
||||
</span><span id="LineageHTML-233"><a href="#LineageHTML-233"><span class="linenos">233</span></a> <span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="LineageHTML-234"><a href="#LineageHTML-234"><span class="linenos">234</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="o">.</span><span class="n">values</span><span class="p">()))</span>
|
||||
</span><span id="LineageHTML-235"><a href="#LineageHTML-235"><span class="linenos">235</span></a> <span class="n">edges</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML-236"><a href="#LineageHTML-236"><span class="linenos">236</span></a> <span class="n">options</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">options</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML-237"><a href="#LineageHTML-237"><span class="linenos">237</span></a> <span class="n">imports</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="LineageHTML-238"><a href="#LineageHTML-238"><span class="linenos">238</span></a><span class="w"> </span><span class="sd">"""<script type="text/javascript" src="https://unpkg.com/vis-data@latest/peer/umd/vis-data.min.js"></script></span>
|
||||
</span><span id="LineageHTML-239"><a href="#LineageHTML-239"><span class="linenos">239</span></a><span class="sd"> <script type="text/javascript" src="https://unpkg.com/vis-network@latest/peer/umd/vis-network.min.js"></script></span>
|
||||
</span><span id="LineageHTML-240"><a href="#LineageHTML-240"><span class="linenos">240</span></a><span class="sd"> <link rel="stylesheet" type="text/css" href="https://unpkg.com/vis-network/styles/vis-network.min.css" />"""</span>
|
||||
</span><span id="LineageHTML-241"><a href="#LineageHTML-241"><span class="linenos">241</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span>
|
||||
</span><span id="LineageHTML-242"><a href="#LineageHTML-242"><span class="linenos">242</span></a> <span class="k">else</span> <span class="s2">""</span>
|
||||
</span><span id="LineageHTML-243"><a href="#LineageHTML-243"><span class="linenos">243</span></a> <span class="p">)</span>
|
||||
</span><span id="LineageHTML-233"><a href="#LineageHTML-233"><span class="linenos">233</span></a> <span class="n">node_id</span> <span class="o">=</span> <span class="nb">id</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML-234"><a href="#LineageHTML-234"><span class="linenos">234</span></a>
|
||||
</span><span id="LineageHTML-235"><a href="#LineageHTML-235"><span class="linenos">235</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="p">[</span><span class="n">node_id</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML-236"><a href="#LineageHTML-236"><span class="linenos">236</span></a> <span class="s2">"id"</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-237"><a href="#LineageHTML-237"><span class="linenos">237</span></a> <span class="s2">"label"</span><span class="p">:</span> <span class="n">label</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-238"><a href="#LineageHTML-238"><span class="linenos">238</span></a> <span class="s2">"title"</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-239"><a href="#LineageHTML-239"><span class="linenos">239</span></a> <span class="s2">"group"</span><span class="p">:</span> <span class="n">group</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML-240"><a href="#LineageHTML-240"><span class="linenos">240</span></a> <span class="p">}</span>
|
||||
</span><span id="LineageHTML-241"><a href="#LineageHTML-241"><span class="linenos">241</span></a>
|
||||
</span><span id="LineageHTML-242"><a href="#LineageHTML-242"><span class="linenos">242</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="p">:</span>
|
||||
</span><span id="LineageHTML-243"><a href="#LineageHTML-243"><span class="linenos">243</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s2">"from"</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> <span class="s2">"to"</span><span class="p">:</span> <span class="nb">id</span><span class="p">(</span><span class="n">d</span><span class="p">)})</span>
|
||||
</span><span id="LineageHTML-244"><a href="#LineageHTML-244"><span class="linenos">244</span></a>
|
||||
</span><span id="LineageHTML-245"><a href="#LineageHTML-245"><span class="linenos">245</span></a> <span class="k">return</span> <span class="sa">f</span><span class="s2">"""<div></span>
|
||||
</span><span id="LineageHTML-246"><a href="#LineageHTML-246"><span class="linenos">246</span></a><span class="s2"> <div id="sqlglot-lineage"></div></span>
|
||||
</span><span id="LineageHTML-247"><a href="#LineageHTML-247"><span class="linenos">247</span></a><span class="s2"> </span><span class="si">{</span><span class="n">imports</span><span class="si">}</span>
|
||||
</span><span id="LineageHTML-248"><a href="#LineageHTML-248"><span class="linenos">248</span></a><span class="s2"> <script type="text/javascript"></span>
|
||||
</span><span id="LineageHTML-249"><a href="#LineageHTML-249"><span class="linenos">249</span></a><span class="s2"> var nodes = new vis.DataSet(</span><span class="si">{</span><span class="n">nodes</span><span class="si">}</span><span class="s2">)</span>
|
||||
</span><span id="LineageHTML-250"><a href="#LineageHTML-250"><span class="linenos">250</span></a><span class="s2"> nodes.forEach(row => row["title"] = new DOMParser().parseFromString(row["title"], "text/html").body.childNodes[0])</span>
|
||||
</span><span id="LineageHTML-251"><a href="#LineageHTML-251"><span class="linenos">251</span></a>
|
||||
</span><span id="LineageHTML-252"><a href="#LineageHTML-252"><span class="linenos">252</span></a><span class="s2"> new vis.Network(</span>
|
||||
</span><span id="LineageHTML-253"><a href="#LineageHTML-253"><span class="linenos">253</span></a><span class="s2"> document.getElementById("sqlglot-lineage"),</span>
|
||||
</span><span id="LineageHTML-254"><a href="#LineageHTML-254"><span class="linenos">254</span></a><span class="s2"> </span><span class="se">{{</span>
|
||||
</span><span id="LineageHTML-255"><a href="#LineageHTML-255"><span class="linenos">255</span></a><span class="s2"> nodes: nodes,</span>
|
||||
</span><span id="LineageHTML-256"><a href="#LineageHTML-256"><span class="linenos">256</span></a><span class="s2"> edges: new vis.DataSet(</span><span class="si">{</span><span class="n">edges</span><span class="si">}</span><span class="s2">)</span>
|
||||
</span><span id="LineageHTML-257"><a href="#LineageHTML-257"><span class="linenos">257</span></a><span class="s2"> </span><span class="se">}}</span><span class="s2">,</span>
|
||||
</span><span id="LineageHTML-258"><a href="#LineageHTML-258"><span class="linenos">258</span></a><span class="s2"> </span><span class="si">{</span><span class="n">options</span><span class="si">}</span><span class="s2">,</span>
|
||||
</span><span id="LineageHTML-259"><a href="#LineageHTML-259"><span class="linenos">259</span></a><span class="s2"> )</span>
|
||||
</span><span id="LineageHTML-260"><a href="#LineageHTML-260"><span class="linenos">260</span></a><span class="s2"> </script></span>
|
||||
</span><span id="LineageHTML-261"><a href="#LineageHTML-261"><span class="linenos">261</span></a><span class="s2"></div>"""</span>
|
||||
</span><span id="LineageHTML-262"><a href="#LineageHTML-262"><span class="linenos">262</span></a>
|
||||
</span><span id="LineageHTML-263"><a href="#LineageHTML-263"><span class="linenos">263</span></a> <span class="k">def</span> <span class="nf">_repr_html_</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="LineageHTML-264"><a href="#LineageHTML-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="fm">__str__</span><span class="p">()</span>
|
||||
</span><span id="LineageHTML-245"><a href="#LineageHTML-245"><span class="linenos">245</span></a> <span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="LineageHTML-246"><a href="#LineageHTML-246"><span class="linenos">246</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="o">.</span><span class="n">values</span><span class="p">()))</span>
|
||||
</span><span id="LineageHTML-247"><a href="#LineageHTML-247"><span class="linenos">247</span></a> <span class="n">edges</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML-248"><a href="#LineageHTML-248"><span class="linenos">248</span></a> <span class="n">options</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">options</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML-249"><a href="#LineageHTML-249"><span class="linenos">249</span></a> <span class="n">imports</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="LineageHTML-250"><a href="#LineageHTML-250"><span class="linenos">250</span></a><span class="w"> </span><span class="sd">"""<script type="text/javascript" src="https://unpkg.com/vis-data@latest/peer/umd/vis-data.min.js"></script></span>
|
||||
</span><span id="LineageHTML-251"><a href="#LineageHTML-251"><span class="linenos">251</span></a><span class="sd"> <script type="text/javascript" src="https://unpkg.com/vis-network@latest/peer/umd/vis-network.min.js"></script></span>
|
||||
</span><span id="LineageHTML-252"><a href="#LineageHTML-252"><span class="linenos">252</span></a><span class="sd"> <link rel="stylesheet" type="text/css" href="https://unpkg.com/vis-network/styles/vis-network.min.css" />"""</span>
|
||||
</span><span id="LineageHTML-253"><a href="#LineageHTML-253"><span class="linenos">253</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span>
|
||||
</span><span id="LineageHTML-254"><a href="#LineageHTML-254"><span class="linenos">254</span></a> <span class="k">else</span> <span class="s2">""</span>
|
||||
</span><span id="LineageHTML-255"><a href="#LineageHTML-255"><span class="linenos">255</span></a> <span class="p">)</span>
|
||||
</span><span id="LineageHTML-256"><a href="#LineageHTML-256"><span class="linenos">256</span></a>
|
||||
</span><span id="LineageHTML-257"><a href="#LineageHTML-257"><span class="linenos">257</span></a> <span class="k">return</span> <span class="sa">f</span><span class="s2">"""<div></span>
|
||||
</span><span id="LineageHTML-258"><a href="#LineageHTML-258"><span class="linenos">258</span></a><span class="s2"> <div id="sqlglot-lineage"></div></span>
|
||||
</span><span id="LineageHTML-259"><a href="#LineageHTML-259"><span class="linenos">259</span></a><span class="s2"> </span><span class="si">{</span><span class="n">imports</span><span class="si">}</span>
|
||||
</span><span id="LineageHTML-260"><a href="#LineageHTML-260"><span class="linenos">260</span></a><span class="s2"> <script type="text/javascript"></span>
|
||||
</span><span id="LineageHTML-261"><a href="#LineageHTML-261"><span class="linenos">261</span></a><span class="s2"> var nodes = new vis.DataSet(</span><span class="si">{</span><span class="n">nodes</span><span class="si">}</span><span class="s2">)</span>
|
||||
</span><span id="LineageHTML-262"><a href="#LineageHTML-262"><span class="linenos">262</span></a><span class="s2"> nodes.forEach(row => row["title"] = new DOMParser().parseFromString(row["title"], "text/html").body.childNodes[0])</span>
|
||||
</span><span id="LineageHTML-263"><a href="#LineageHTML-263"><span class="linenos">263</span></a>
|
||||
</span><span id="LineageHTML-264"><a href="#LineageHTML-264"><span class="linenos">264</span></a><span class="s2"> new vis.Network(</span>
|
||||
</span><span id="LineageHTML-265"><a href="#LineageHTML-265"><span class="linenos">265</span></a><span class="s2"> document.getElementById("sqlglot-lineage"),</span>
|
||||
</span><span id="LineageHTML-266"><a href="#LineageHTML-266"><span class="linenos">266</span></a><span class="s2"> </span><span class="se">{{</span>
|
||||
</span><span id="LineageHTML-267"><a href="#LineageHTML-267"><span class="linenos">267</span></a><span class="s2"> nodes: nodes,</span>
|
||||
</span><span id="LineageHTML-268"><a href="#LineageHTML-268"><span class="linenos">268</span></a><span class="s2"> edges: new vis.DataSet(</span><span class="si">{</span><span class="n">edges</span><span class="si">}</span><span class="s2">)</span>
|
||||
</span><span id="LineageHTML-269"><a href="#LineageHTML-269"><span class="linenos">269</span></a><span class="s2"> </span><span class="se">}}</span><span class="s2">,</span>
|
||||
</span><span id="LineageHTML-270"><a href="#LineageHTML-270"><span class="linenos">270</span></a><span class="s2"> </span><span class="si">{</span><span class="n">options</span><span class="si">}</span><span class="s2">,</span>
|
||||
</span><span id="LineageHTML-271"><a href="#LineageHTML-271"><span class="linenos">271</span></a><span class="s2"> )</span>
|
||||
</span><span id="LineageHTML-272"><a href="#LineageHTML-272"><span class="linenos">272</span></a><span class="s2"> </script></span>
|
||||
</span><span id="LineageHTML-273"><a href="#LineageHTML-273"><span class="linenos">273</span></a><span class="s2"></div>"""</span>
|
||||
</span><span id="LineageHTML-274"><a href="#LineageHTML-274"><span class="linenos">274</span></a>
|
||||
</span><span id="LineageHTML-275"><a href="#LineageHTML-275"><span class="linenos">275</span></a> <span class="k">def</span> <span class="nf">_repr_html_</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="LineageHTML-276"><a href="#LineageHTML-276"><span class="linenos">276</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="fm">__str__</span><span class="p">()</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -818,76 +842,76 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#LineageHTML.__init__"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="LineageHTML.__init__-162"><a href="#LineageHTML.__init__-162"><span class="linenos">162</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span>
|
||||
</span><span id="LineageHTML.__init__-163"><a href="#LineageHTML.__init__-163"><span class="linenos">163</span></a> <span class="bp">self</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-164"><a href="#LineageHTML.__init__-164"><span class="linenos">164</span></a> <span class="n">node</span><span class="p">:</span> <span class="n">Node</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-165"><a href="#LineageHTML.__init__-165"><span class="linenos">165</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-166"><a href="#LineageHTML.__init__-166"><span class="linenos">166</span></a> <span class="n">imports</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-167"><a href="#LineageHTML.__init__-167"><span class="linenos">167</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-168"><a href="#LineageHTML.__init__-168"><span class="linenos">168</span></a> <span class="p">):</span>
|
||||
</span><span id="LineageHTML.__init__-169"><a href="#LineageHTML.__init__-169"><span class="linenos">169</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">node</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="LineageHTML.__init__-170"><a href="#LineageHTML.__init__-170"><span class="linenos">170</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span> <span class="o">=</span> <span class="n">imports</span>
|
||||
</span><span id="LineageHTML.__init__-171"><a href="#LineageHTML.__init__-171"><span class="linenos">171</span></a>
|
||||
</span><span id="LineageHTML.__init__-172"><a href="#LineageHTML.__init__-172"><span class="linenos">172</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-173"><a href="#LineageHTML.__init__-173"><span class="linenos">173</span></a> <span class="s2">"height"</span><span class="p">:</span> <span class="s2">"500px"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-174"><a href="#LineageHTML.__init__-174"><span class="linenos">174</span></a> <span class="s2">"width"</span><span class="p">:</span> <span class="s2">"100%"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-175"><a href="#LineageHTML.__init__-175"><span class="linenos">175</span></a> <span class="s2">"layout"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-176"><a href="#LineageHTML.__init__-176"><span class="linenos">176</span></a> <span class="s2">"hierarchical"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-177"><a href="#LineageHTML.__init__-177"><span class="linenos">177</span></a> <span class="s2">"enabled"</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-178"><a href="#LineageHTML.__init__-178"><span class="linenos">178</span></a> <span class="s2">"nodeSpacing"</span><span class="p">:</span> <span class="mi">200</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-179"><a href="#LineageHTML.__init__-179"><span class="linenos">179</span></a> <span class="s2">"sortMethod"</span><span class="p">:</span> <span class="s2">"directed"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-180"><a href="#LineageHTML.__init__-180"><span class="linenos">180</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML.__init__-181"><a href="#LineageHTML.__init__-181"><span class="linenos">181</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML.__init__-182"><a href="#LineageHTML.__init__-182"><span class="linenos">182</span></a> <span class="s2">"interaction"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-183"><a href="#LineageHTML.__init__-183"><span class="linenos">183</span></a> <span class="s2">"dragNodes"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-184"><a href="#LineageHTML.__init__-184"><span class="linenos">184</span></a> <span class="s2">"selectable"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-185"><a href="#LineageHTML.__init__-185"><span class="linenos">185</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML.__init__-186"><a href="#LineageHTML.__init__-186"><span class="linenos">186</span></a> <span class="s2">"physics"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-187"><a href="#LineageHTML.__init__-187"><span class="linenos">187</span></a> <span class="s2">"enabled"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-188"><a href="#LineageHTML.__init__-188"><span class="linenos">188</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML.__init__-189"><a href="#LineageHTML.__init__-189"><span class="linenos">189</span></a> <span class="s2">"edges"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-190"><a href="#LineageHTML.__init__-190"><span class="linenos">190</span></a> <span class="s2">"arrows"</span><span class="p">:</span> <span class="s2">"to"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-191"><a href="#LineageHTML.__init__-191"><span class="linenos">191</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML.__init__-192"><a href="#LineageHTML.__init__-192"><span class="linenos">192</span></a> <span class="s2">"nodes"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-193"><a href="#LineageHTML.__init__-193"><span class="linenos">193</span></a> <span class="s2">"font"</span><span class="p">:</span> <span class="s2">"20px monaco"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-194"><a href="#LineageHTML.__init__-194"><span class="linenos">194</span></a> <span class="s2">"shape"</span><span class="p">:</span> <span class="s2">"box"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-195"><a href="#LineageHTML.__init__-195"><span class="linenos">195</span></a> <span class="s2">"widthConstraint"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-196"><a href="#LineageHTML.__init__-196"><span class="linenos">196</span></a> <span class="s2">"maximum"</span><span class="p">:</span> <span class="mi">300</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-197"><a href="#LineageHTML.__init__-197"><span class="linenos">197</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML.__init__-198"><a href="#LineageHTML.__init__-198"><span class="linenos">198</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML.__init__-199"><a href="#LineageHTML.__init__-199"><span class="linenos">199</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-200"><a href="#LineageHTML.__init__-200"><span class="linenos">200</span></a> <span class="p">}</span>
|
||||
</span><span id="LineageHTML.__init__-201"><a href="#LineageHTML.__init__-201"><span class="linenos">201</span></a>
|
||||
</span><span id="LineageHTML.__init__-202"><a href="#LineageHTML.__init__-202"><span class="linenos">202</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="LineageHTML.__init__-203"><a href="#LineageHTML.__init__-203"><span class="linenos">203</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="LineageHTML.__init__-204"><a href="#LineageHTML.__init__-204"><span class="linenos">204</span></a>
|
||||
</span><span id="LineageHTML.__init__-205"><a href="#LineageHTML.__init__-205"><span class="linenos">205</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
|
||||
</span><span id="LineageHTML.__init__-206"><a href="#LineageHTML.__init__-206"><span class="linenos">206</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="LineageHTML.__init__-207"><a href="#LineageHTML.__init__-207"><span class="linenos">207</span></a> <span class="n">label</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">"</span>
|
||||
</span><span id="LineageHTML.__init__-208"><a href="#LineageHTML.__init__-208"><span class="linenos">208</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"<pre>SELECT </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2"> FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2"></pre>"</span>
|
||||
</span><span id="LineageHTML.__init__-209"><a href="#LineageHTML.__init__-209"><span class="linenos">209</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">1</span>
|
||||
</span><span id="LineageHTML.__init__-210"><a href="#LineageHTML.__init__-210"><span class="linenos">210</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="LineageHTML.__init__-211"><a href="#LineageHTML.__init__-211"><span class="linenos">211</span></a> <span class="n">label</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML.__init__-212"><a href="#LineageHTML.__init__-212"><span class="linenos">212</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">source</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
|
||||
</span><span id="LineageHTML.__init__-213"><a href="#LineageHTML.__init__-213"><span class="linenos">213</span></a> <span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Tag</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">n</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">"<b>"</span><span class="p">,</span> <span class="n">postfix</span><span class="o">=</span><span class="s2">"</b>"</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML.__init__-214"><a href="#LineageHTML.__init__-214"><span class="linenos">214</span></a> <span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="LineageHTML.__init__-215"><a href="#LineageHTML.__init__-215"><span class="linenos">215</span></a> <span class="k">else</span> <span class="n">n</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-216"><a href="#LineageHTML.__init__-216"><span class="linenos">216</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-217"><a href="#LineageHTML.__init__-217"><span class="linenos">217</span></a> <span class="p">)</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML.__init__-218"><a href="#LineageHTML.__init__-218"><span class="linenos">218</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"<pre></span><span class="si">{</span><span class="n">source</span><span class="si">}</span><span class="s2"></pre>"</span>
|
||||
</span><span id="LineageHTML.__init__-219"><a href="#LineageHTML.__init__-219"><span class="linenos">219</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
</span><span id="LineageHTML.__init__-220"><a href="#LineageHTML.__init__-220"><span class="linenos">220</span></a>
|
||||
</span><span id="LineageHTML.__init__-221"><a href="#LineageHTML.__init__-221"><span class="linenos">221</span></a> <span class="n">node_id</span> <span class="o">=</span> <span class="nb">id</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML.__init__-222"><a href="#LineageHTML.__init__-222"><span class="linenos">222</span></a>
|
||||
</span><span id="LineageHTML.__init__-223"><a href="#LineageHTML.__init__-223"><span class="linenos">223</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="p">[</span><span class="n">node_id</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-224"><a href="#LineageHTML.__init__-224"><span class="linenos">224</span></a> <span class="s2">"id"</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-225"><a href="#LineageHTML.__init__-225"><span class="linenos">225</span></a> <span class="s2">"label"</span><span class="p">:</span> <span class="n">label</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-226"><a href="#LineageHTML.__init__-226"><span class="linenos">226</span></a> <span class="s2">"title"</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-227"><a href="#LineageHTML.__init__-227"><span class="linenos">227</span></a> <span class="s2">"group"</span><span class="p">:</span> <span class="n">group</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-228"><a href="#LineageHTML.__init__-228"><span class="linenos">228</span></a> <span class="p">}</span>
|
||||
</span><span id="LineageHTML.__init__-229"><a href="#LineageHTML.__init__-229"><span class="linenos">229</span></a>
|
||||
</span><span id="LineageHTML.__init__-230"><a href="#LineageHTML.__init__-230"><span class="linenos">230</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="p">:</span>
|
||||
</span><span id="LineageHTML.__init__-231"><a href="#LineageHTML.__init__-231"><span class="linenos">231</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s2">"from"</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> <span class="s2">"to"</span><span class="p">:</span> <span class="nb">id</span><span class="p">(</span><span class="n">d</span><span class="p">)})</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="LineageHTML.__init__-174"><a href="#LineageHTML.__init__-174"><span class="linenos">174</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span>
|
||||
</span><span id="LineageHTML.__init__-175"><a href="#LineageHTML.__init__-175"><span class="linenos">175</span></a> <span class="bp">self</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-176"><a href="#LineageHTML.__init__-176"><span class="linenos">176</span></a> <span class="n">node</span><span class="p">:</span> <span class="n">Node</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-177"><a href="#LineageHTML.__init__-177"><span class="linenos">177</span></a> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-178"><a href="#LineageHTML.__init__-178"><span class="linenos">178</span></a> <span class="n">imports</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-179"><a href="#LineageHTML.__init__-179"><span class="linenos">179</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-180"><a href="#LineageHTML.__init__-180"><span class="linenos">180</span></a> <span class="p">):</span>
|
||||
</span><span id="LineageHTML.__init__-181"><a href="#LineageHTML.__init__-181"><span class="linenos">181</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">node</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="LineageHTML.__init__-182"><a href="#LineageHTML.__init__-182"><span class="linenos">182</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">imports</span> <span class="o">=</span> <span class="n">imports</span>
|
||||
</span><span id="LineageHTML.__init__-183"><a href="#LineageHTML.__init__-183"><span class="linenos">183</span></a>
|
||||
</span><span id="LineageHTML.__init__-184"><a href="#LineageHTML.__init__-184"><span class="linenos">184</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-185"><a href="#LineageHTML.__init__-185"><span class="linenos">185</span></a> <span class="s2">"height"</span><span class="p">:</span> <span class="s2">"500px"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-186"><a href="#LineageHTML.__init__-186"><span class="linenos">186</span></a> <span class="s2">"width"</span><span class="p">:</span> <span class="s2">"100%"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-187"><a href="#LineageHTML.__init__-187"><span class="linenos">187</span></a> <span class="s2">"layout"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-188"><a href="#LineageHTML.__init__-188"><span class="linenos">188</span></a> <span class="s2">"hierarchical"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-189"><a href="#LineageHTML.__init__-189"><span class="linenos">189</span></a> <span class="s2">"enabled"</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-190"><a href="#LineageHTML.__init__-190"><span class="linenos">190</span></a> <span class="s2">"nodeSpacing"</span><span class="p">:</span> <span class="mi">200</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-191"><a href="#LineageHTML.__init__-191"><span class="linenos">191</span></a> <span class="s2">"sortMethod"</span><span class="p">:</span> <span class="s2">"directed"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-192"><a href="#LineageHTML.__init__-192"><span class="linenos">192</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML.__init__-193"><a href="#LineageHTML.__init__-193"><span class="linenos">193</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML.__init__-194"><a href="#LineageHTML.__init__-194"><span class="linenos">194</span></a> <span class="s2">"interaction"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-195"><a href="#LineageHTML.__init__-195"><span class="linenos">195</span></a> <span class="s2">"dragNodes"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-196"><a href="#LineageHTML.__init__-196"><span class="linenos">196</span></a> <span class="s2">"selectable"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-197"><a href="#LineageHTML.__init__-197"><span class="linenos">197</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML.__init__-198"><a href="#LineageHTML.__init__-198"><span class="linenos">198</span></a> <span class="s2">"physics"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-199"><a href="#LineageHTML.__init__-199"><span class="linenos">199</span></a> <span class="s2">"enabled"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-200"><a href="#LineageHTML.__init__-200"><span class="linenos">200</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML.__init__-201"><a href="#LineageHTML.__init__-201"><span class="linenos">201</span></a> <span class="s2">"edges"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-202"><a href="#LineageHTML.__init__-202"><span class="linenos">202</span></a> <span class="s2">"arrows"</span><span class="p">:</span> <span class="s2">"to"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-203"><a href="#LineageHTML.__init__-203"><span class="linenos">203</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML.__init__-204"><a href="#LineageHTML.__init__-204"><span class="linenos">204</span></a> <span class="s2">"nodes"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-205"><a href="#LineageHTML.__init__-205"><span class="linenos">205</span></a> <span class="s2">"font"</span><span class="p">:</span> <span class="s2">"20px monaco"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-206"><a href="#LineageHTML.__init__-206"><span class="linenos">206</span></a> <span class="s2">"shape"</span><span class="p">:</span> <span class="s2">"box"</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-207"><a href="#LineageHTML.__init__-207"><span class="linenos">207</span></a> <span class="s2">"widthConstraint"</span><span class="p">:</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-208"><a href="#LineageHTML.__init__-208"><span class="linenos">208</span></a> <span class="s2">"maximum"</span><span class="p">:</span> <span class="mi">300</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-209"><a href="#LineageHTML.__init__-209"><span class="linenos">209</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML.__init__-210"><a href="#LineageHTML.__init__-210"><span class="linenos">210</span></a> <span class="p">},</span>
|
||||
</span><span id="LineageHTML.__init__-211"><a href="#LineageHTML.__init__-211"><span class="linenos">211</span></a> <span class="o">**</span><span class="n">opts</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-212"><a href="#LineageHTML.__init__-212"><span class="linenos">212</span></a> <span class="p">}</span>
|
||||
</span><span id="LineageHTML.__init__-213"><a href="#LineageHTML.__init__-213"><span class="linenos">213</span></a>
|
||||
</span><span id="LineageHTML.__init__-214"><a href="#LineageHTML.__init__-214"><span class="linenos">214</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="LineageHTML.__init__-215"><a href="#LineageHTML.__init__-215"><span class="linenos">215</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="LineageHTML.__init__-216"><a href="#LineageHTML.__init__-216"><span class="linenos">216</span></a>
|
||||
</span><span id="LineageHTML.__init__-217"><a href="#LineageHTML.__init__-217"><span class="linenos">217</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">walk</span><span class="p">():</span>
|
||||
</span><span id="LineageHTML.__init__-218"><a href="#LineageHTML.__init__-218"><span class="linenos">218</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="LineageHTML.__init__-219"><a href="#LineageHTML.__init__-219"><span class="linenos">219</span></a> <span class="n">label</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2">"</span>
|
||||
</span><span id="LineageHTML.__init__-220"><a href="#LineageHTML.__init__-220"><span class="linenos">220</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"<pre>SELECT </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2"> FROM </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="si">}</span><span class="s2"></pre>"</span>
|
||||
</span><span id="LineageHTML.__init__-221"><a href="#LineageHTML.__init__-221"><span class="linenos">221</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">1</span>
|
||||
</span><span id="LineageHTML.__init__-222"><a href="#LineageHTML.__init__-222"><span class="linenos">222</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="LineageHTML.__init__-223"><a href="#LineageHTML.__init__-223"><span class="linenos">223</span></a> <span class="n">label</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML.__init__-224"><a href="#LineageHTML.__init__-224"><span class="linenos">224</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">source</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
|
||||
</span><span id="LineageHTML.__init__-225"><a href="#LineageHTML.__init__-225"><span class="linenos">225</span></a> <span class="k">lambda</span> <span class="n">n</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Tag</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">n</span><span class="p">,</span> <span class="n">prefix</span><span class="o">=</span><span class="s2">"<b>"</span><span class="p">,</span> <span class="n">postfix</span><span class="o">=</span><span class="s2">"</b>"</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML.__init__-226"><a href="#LineageHTML.__init__-226"><span class="linenos">226</span></a> <span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="n">node</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="LineageHTML.__init__-227"><a href="#LineageHTML.__init__-227"><span class="linenos">227</span></a> <span class="k">else</span> <span class="n">n</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-228"><a href="#LineageHTML.__init__-228"><span class="linenos">228</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-229"><a href="#LineageHTML.__init__-229"><span class="linenos">229</span></a> <span class="p">)</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML.__init__-230"><a href="#LineageHTML.__init__-230"><span class="linenos">230</span></a> <span class="n">title</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"<pre></span><span class="si">{</span><span class="n">source</span><span class="si">}</span><span class="s2"></pre>"</span>
|
||||
</span><span id="LineageHTML.__init__-231"><a href="#LineageHTML.__init__-231"><span class="linenos">231</span></a> <span class="n">group</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
</span><span id="LineageHTML.__init__-232"><a href="#LineageHTML.__init__-232"><span class="linenos">232</span></a>
|
||||
</span><span id="LineageHTML.__init__-233"><a href="#LineageHTML.__init__-233"><span class="linenos">233</span></a> <span class="n">node_id</span> <span class="o">=</span> <span class="nb">id</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="LineageHTML.__init__-234"><a href="#LineageHTML.__init__-234"><span class="linenos">234</span></a>
|
||||
</span><span id="LineageHTML.__init__-235"><a href="#LineageHTML.__init__-235"><span class="linenos">235</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">nodes</span><span class="p">[</span><span class="n">node_id</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="LineageHTML.__init__-236"><a href="#LineageHTML.__init__-236"><span class="linenos">236</span></a> <span class="s2">"id"</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-237"><a href="#LineageHTML.__init__-237"><span class="linenos">237</span></a> <span class="s2">"label"</span><span class="p">:</span> <span class="n">label</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-238"><a href="#LineageHTML.__init__-238"><span class="linenos">238</span></a> <span class="s2">"title"</span><span class="p">:</span> <span class="n">title</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-239"><a href="#LineageHTML.__init__-239"><span class="linenos">239</span></a> <span class="s2">"group"</span><span class="p">:</span> <span class="n">group</span><span class="p">,</span>
|
||||
</span><span id="LineageHTML.__init__-240"><a href="#LineageHTML.__init__-240"><span class="linenos">240</span></a> <span class="p">}</span>
|
||||
</span><span id="LineageHTML.__init__-241"><a href="#LineageHTML.__init__-241"><span class="linenos">241</span></a>
|
||||
</span><span id="LineageHTML.__init__-242"><a href="#LineageHTML.__init__-242"><span class="linenos">242</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">downstream</span><span class="p">:</span>
|
||||
</span><span id="LineageHTML.__init__-243"><a href="#LineageHTML.__init__-243"><span class="linenos">243</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">edges</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s2">"from"</span><span class="p">:</span> <span class="n">node_id</span><span class="p">,</span> <span class="s2">"to"</span><span class="p">:</span> <span class="nb">id</span><span class="p">(</span><span class="n">d</span><span class="p">)})</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -202,60 +202,61 @@
|
|||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span> <span class="o">=</span> <span class="n">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="n">table</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">table_</span><span class="p">(</span><span class="n">name</span><span class="p">),</span> <span class="n">alias</span><span class="o">=</span><span class="n">parent</span><span class="o">.</span><span class="n">alias</span> <span class="ow">or</span> <span class="n">name</span><span class="p">)</span>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">parent</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">))</span>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="k">return</span> <span class="n">cte</span>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">table</span><span class="p">)</span>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="k">return</span> <span class="n">cte</span>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a><span class="k">def</span> <span class="nf">_eliminate_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span> <span class="o">=</span> <span class="n">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">parent</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">with_</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">with_</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="c1"># Rename references to this CTE</span>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">for</span> <span class="n">child_scope</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">traverse</span><span class="p">():</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">child_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="k">if</span> <span class="n">source</span> <span class="ow">is</span> <span class="n">scope</span><span class="p">:</span>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="n">new_table</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">table_</span><span class="p">(</span><span class="n">name</span><span class="p">),</span> <span class="n">alias</span><span class="o">=</span><span class="n">table</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="n">table</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">new_table</span><span class="p">)</span>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="k">return</span> <span class="n">cte</span>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a><span class="k">def</span> <span class="nf">_eliminate_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span> <span class="o">=</span> <span class="n">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">)</span>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">parent</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">with_</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="n">with_</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="c1"># Rename references to this CTE</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="k">for</span> <span class="n">child_scope</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">traverse</span><span class="p">():</span>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">child_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="k">if</span> <span class="n">source</span> <span class="ow">is</span> <span class="n">scope</span><span class="p">:</span>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="n">new_table</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">table_</span><span class="p">(</span><span class="n">name</span><span class="p">),</span> <span class="n">alias</span><span class="o">=</span><span class="n">table</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="n">table</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">new_table</span><span class="p">)</span>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="k">return</span> <span class="n">cte</span>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a><span class="k">def</span> <span class="nf">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a><span class="sd"> tuple of (name, cte)</span>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a><span class="sd"> where `name` is a new name for this CTE in the root scope and `cte` is a new CTE instance.</span>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a><span class="sd"> If this CTE duplicates an existing CTE, `cte` will be None.</span>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="n">duplicate_cte_alias</span> <span class="o">=</span> <span class="n">existing_ctes</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">parent</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="o">=</span><span class="n">taken</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="s2">"cte"</span><span class="p">)</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="k">if</span> <span class="n">duplicate_cte_alias</span><span class="p">:</span>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">duplicate_cte_alias</span>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="k">elif</span> <span class="n">taken</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="o">=</span><span class="n">taken</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="n">name</span><span class="p">)</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="n">taken</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">scope</span>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">duplicate_cte_alias</span><span class="p">:</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="n">existing_ctes</span><span class="p">[</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">]</span> <span class="o">=</span> <span class="n">name</span>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">CTE</span><span class="p">(</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="n">this</span><span class="o">=</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">name</span><span class="p">)),</span>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="p">)</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="k">return</span> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a><span class="k">def</span> <span class="nf">_new_cte</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">existing_ctes</span><span class="p">,</span> <span class="n">taken</span><span class="p">):</span>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a><span class="sd"> tuple of (name, cte)</span>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a><span class="sd"> where `name` is a new name for this CTE in the root scope and `cte` is a new CTE instance.</span>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a><span class="sd"> If this CTE duplicates an existing CTE, `cte` will be None.</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="n">duplicate_cte_alias</span> <span class="o">=</span> <span class="n">existing_ctes</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">parent</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="o">=</span><span class="n">taken</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="s2">"cte"</span><span class="p">)</span>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="k">if</span> <span class="n">duplicate_cte_alias</span><span class="p">:</span>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">duplicate_cte_alias</span>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="k">elif</span> <span class="n">taken</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="o">=</span><span class="n">taken</span><span class="p">,</span> <span class="n">base</span><span class="o">=</span><span class="n">name</span><span class="p">)</span>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="n">taken</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">scope</span>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">duplicate_cte_alias</span><span class="p">:</span>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="n">existing_ctes</span><span class="p">[</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">]</span> <span class="o">=</span> <span class="n">name</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">CTE</span><span class="p">(</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="n">this</span><span class="o">=</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="n">alias</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">name</span><span class="p">)),</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="p">)</span>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="k">return</span> <span class="n">name</span><span class="p">,</span> <span class="n">cte</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -246,228 +246,230 @@
|
|||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="k">return</span> <span class="p">(</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">inner_select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="n">UNMERGABLE_ARGS</span><span class="p">)</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">)</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">pivots</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span><span class="n">leave_tables_isolated</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span> <span class="o">></span> <span class="mi">1</span><span class="p">)</span>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">)</span>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">)</span>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="ow">and</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">side</span> <span class="ow">in</span> <span class="p">{</span><span class="s2">"FULL"</span><span class="p">,</span> <span class="s2">"LEFT"</span><span class="p">,</span> <span class="s2">"RIGHT"</span><span class="p">}</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="p">)</span>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">)</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="ow">and</span> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="n">j</span><span class="o">.</span><span class="n">side</span> <span class="ow">in</span> <span class="p">{</span><span class="s2">"FULL"</span><span class="p">,</span> <span class="s2">"RIGHT"</span><span class="p">}</span> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="p">)</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="p">)</span>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">_outer_select_joins_on_inner_select_join</span><span class="p">()</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">_is_a_window_expression_in_unmergable_operation</span><span class="p">()</span>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="p">)</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">is_star</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">inner_select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="n">UNMERGABLE_ARGS</span><span class="p">)</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">)</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">pivots</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span><span class="n">leave_tables_isolated</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span> <span class="o">></span> <span class="mi">1</span><span class="p">)</span>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">)</span>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">)</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="ow">and</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">side</span> <span class="ow">in</span> <span class="p">{</span><span class="s2">"FULL"</span><span class="p">,</span> <span class="s2">"LEFT"</span><span class="p">,</span> <span class="s2">"RIGHT"</span><span class="p">}</span>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="p">)</span>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">)</span>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="ow">and</span> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="n">j</span><span class="o">.</span><span class="n">side</span> <span class="ow">in</span> <span class="p">{</span><span class="s2">"FULL"</span><span class="p">,</span> <span class="s2">"RIGHT"</span><span class="p">}</span> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="p">)</span>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="p">)</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">_outer_select_joins_on_inner_select_join</span><span class="p">()</span>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">_is_a_window_expression_in_unmergable_operation</span><span class="p">()</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="p">)</span>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a>
|
||||
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a><span class="k">def</span> <span class="nf">_rename_inner_sources</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a><span class="sd"> Renames any sources in the inner query that conflict with names in the outer query.</span>
|
||||
</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a>
|
||||
</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a><span class="sd"> alias (str)</span>
|
||||
</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="n">taken</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span>
|
||||
</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a> <span class="n">conflicts</span> <span class="o">=</span> <span class="n">taken</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">inner_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">))</span>
|
||||
</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="n">conflicts</span> <span class="o">-=</span> <span class="p">{</span><span class="n">alias</span><span class="p">}</span>
|
||||
</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a>
|
||||
</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a> <span class="k">for</span> <span class="n">conflict</span> <span class="ow">in</span> <span class="n">conflicts</span><span class="p">:</span>
|
||||
</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a> <span class="n">new_name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="p">,</span> <span class="n">conflict</span><span class="p">)</span>
|
||||
</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a>
|
||||
</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a> <span class="n">source</span><span class="p">,</span> <span class="n">_</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">conflict</span><span class="p">]</span>
|
||||
</span><span id="L-219"><a href="#L-219"><span class="linenos">219</span></a> <span class="n">new_alias</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_name</span><span class="p">)</span>
|
||||
</span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a>
|
||||
</span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
|
||||
</span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">new_alias</span><span class="p">))</span>
|
||||
</span><span id="L-223"><a href="#L-223"><span class="linenos">223</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-224"><a href="#L-224"><span class="linenos">224</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">new_alias</span><span class="p">)</span>
|
||||
</span><span id="L-225"><a href="#L-225"><span class="linenos">225</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-226"><a href="#L-226"><span class="linenos">226</span></a> <span class="n">source</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">new_alias</span><span class="p">))</span>
|
||||
</span><span id="L-227"><a href="#L-227"><span class="linenos">227</span></a>
|
||||
</span><span id="L-228"><a href="#L-228"><span class="linenos">228</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">source_columns</span><span class="p">(</span><span class="n">conflict</span><span class="p">):</span>
|
||||
</span><span id="L-229"><a href="#L-229"><span class="linenos">229</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"table"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_name</span><span class="p">))</span>
|
||||
</span><span id="L-230"><a href="#L-230"><span class="linenos">230</span></a>
|
||||
</span><span id="L-231"><a href="#L-231"><span class="linenos">231</span></a> <span class="n">inner_scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="n">conflict</span><span class="p">,</span> <span class="n">new_name</span><span class="p">)</span>
|
||||
</span><span id="L-232"><a href="#L-232"><span class="linenos">232</span></a>
|
||||
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a>
|
||||
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a><span class="k">def</span> <span class="nf">_rename_inner_sources</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a><span class="sd"> Renames any sources in the inner query that conflict with names in the outer query.</span>
|
||||
</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a>
|
||||
</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a><span class="sd"> alias (str)</span>
|
||||
</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a> <span class="n">taken</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span>
|
||||
</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="n">conflicts</span> <span class="o">=</span> <span class="n">taken</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">inner_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">))</span>
|
||||
</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a> <span class="n">conflicts</span> <span class="o">-=</span> <span class="p">{</span><span class="n">alias</span><span class="p">}</span>
|
||||
</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a>
|
||||
</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a> <span class="k">for</span> <span class="n">conflict</span> <span class="ow">in</span> <span class="n">conflicts</span><span class="p">:</span>
|
||||
</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a> <span class="n">new_name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="p">,</span> <span class="n">conflict</span><span class="p">)</span>
|
||||
</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a>
|
||||
</span><span id="L-219"><a href="#L-219"><span class="linenos">219</span></a> <span class="n">source</span><span class="p">,</span> <span class="n">_</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">conflict</span><span class="p">]</span>
|
||||
</span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a> <span class="n">new_alias</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_name</span><span class="p">)</span>
|
||||
</span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a>
|
||||
</span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
|
||||
</span><span id="L-223"><a href="#L-223"><span class="linenos">223</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">new_alias</span><span class="p">))</span>
|
||||
</span><span id="L-224"><a href="#L-224"><span class="linenos">224</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-225"><a href="#L-225"><span class="linenos">225</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">new_alias</span><span class="p">)</span>
|
||||
</span><span id="L-226"><a href="#L-226"><span class="linenos">226</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-227"><a href="#L-227"><span class="linenos">227</span></a> <span class="n">source</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">new_alias</span><span class="p">))</span>
|
||||
</span><span id="L-228"><a href="#L-228"><span class="linenos">228</span></a>
|
||||
</span><span id="L-229"><a href="#L-229"><span class="linenos">229</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">source_columns</span><span class="p">(</span><span class="n">conflict</span><span class="p">):</span>
|
||||
</span><span id="L-230"><a href="#L-230"><span class="linenos">230</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"table"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_name</span><span class="p">))</span>
|
||||
</span><span id="L-231"><a href="#L-231"><span class="linenos">231</span></a>
|
||||
</span><span id="L-232"><a href="#L-232"><span class="linenos">232</span></a> <span class="n">inner_scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="n">conflict</span><span class="p">,</span> <span class="n">new_name</span><span class="p">)</span>
|
||||
</span><span id="L-233"><a href="#L-233"><span class="linenos">233</span></a>
|
||||
</span><span id="L-234"><a href="#L-234"><span class="linenos">234</span></a><span class="k">def</span> <span class="nf">_merge_from</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">node_to_replace</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-235"><a href="#L-235"><span class="linenos">235</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-236"><a href="#L-236"><span class="linenos">236</span></a><span class="sd"> Merge FROM clause of inner query into outer query.</span>
|
||||
</span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a>
|
||||
</span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a><span class="sd"> node_to_replace (exp.Subquery|exp.Table)</span>
|
||||
</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a><span class="sd"> alias (str)</span>
|
||||
</span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="n">new_subquery</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"from"</span><span class="p">]</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a> <span class="n">node_to_replace</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">new_subquery</span><span class="p">)</span>
|
||||
</span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a> <span class="k">for</span> <span class="n">join_hint</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">join_hints</span><span class="p">:</span>
|
||||
</span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">join_hint</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
|
||||
</span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">:</span>
|
||||
</span><span id="L-249"><a href="#L-249"><span class="linenos">249</span></a> <span class="k">if</span> <span class="n">table</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">node_to_replace</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span>
|
||||
</span><span id="L-250"><a href="#L-250"><span class="linenos">250</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"this"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_subquery</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">))</span>
|
||||
</span><span id="L-251"><a href="#L-251"><span class="linenos">251</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">remove_source</span><span class="p">(</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="L-252"><a href="#L-252"><span class="linenos">252</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">add_source</span><span class="p">(</span>
|
||||
</span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a> <span class="n">new_subquery</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">new_subquery</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">]</span>
|
||||
</span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a> <span class="p">)</span>
|
||||
</span><span id="L-255"><a href="#L-255"><span class="linenos">255</span></a>
|
||||
</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a>
|
||||
</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a><span class="k">def</span> <span class="nf">_merge_joins</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">):</span>
|
||||
</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a><span class="sd"> Merge JOIN clauses of inner query into outer query.</span>
|
||||
</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a>
|
||||
</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a><span class="sd"> from_or_join (exp.From|exp.Join)</span>
|
||||
</span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a>
|
||||
</span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a> <span class="n">new_joins</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-234"><a href="#L-234"><span class="linenos">234</span></a>
|
||||
</span><span id="L-235"><a href="#L-235"><span class="linenos">235</span></a><span class="k">def</span> <span class="nf">_merge_from</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">node_to_replace</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-236"><a href="#L-236"><span class="linenos">236</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a><span class="sd"> Merge FROM clause of inner query into outer query.</span>
|
||||
</span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a>
|
||||
</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a><span class="sd"> node_to_replace (exp.Subquery|exp.Table)</span>
|
||||
</span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a><span class="sd"> alias (str)</span>
|
||||
</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a> <span class="n">new_subquery</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"from"</span><span class="p">]</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a> <span class="n">new_subquery</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">node_to_replace</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">))</span>
|
||||
</span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a> <span class="n">node_to_replace</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">new_subquery</span><span class="p">)</span>
|
||||
</span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a> <span class="k">for</span> <span class="n">join_hint</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">join_hints</span><span class="p">:</span>
|
||||
</span><span id="L-249"><a href="#L-249"><span class="linenos">249</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">join_hint</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
|
||||
</span><span id="L-250"><a href="#L-250"><span class="linenos">250</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">:</span>
|
||||
</span><span id="L-251"><a href="#L-251"><span class="linenos">251</span></a> <span class="k">if</span> <span class="n">table</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">node_to_replace</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span>
|
||||
</span><span id="L-252"><a href="#L-252"><span class="linenos">252</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"this"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_subquery</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">))</span>
|
||||
</span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">remove_source</span><span class="p">(</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">add_source</span><span class="p">(</span>
|
||||
</span><span id="L-255"><a href="#L-255"><span class="linenos">255</span></a> <span class="n">new_subquery</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">new_subquery</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">]</span>
|
||||
</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a> <span class="p">)</span>
|
||||
</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a>
|
||||
</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a>
|
||||
</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a><span class="k">def</span> <span class="nf">_merge_joins</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">):</span>
|
||||
</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a><span class="sd"> Merge JOIN clauses of inner query into outer query.</span>
|
||||
</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a>
|
||||
</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a><span class="sd"> from_or_join (exp.From|exp.Join)</span>
|
||||
</span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-268"><a href="#L-268"><span class="linenos">268</span></a>
|
||||
</span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="p">:</span>
|
||||
</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a> <span class="n">new_joins</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">add_source</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">])</span>
|
||||
</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a>
|
||||
</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a> <span class="k">if</span> <span class="n">new_joins</span><span class="p">:</span>
|
||||
</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a> <span class="n">outer_joins</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a>
|
||||
</span><span id="L-277"><a href="#L-277"><span class="linenos">277</span></a> <span class="c1"># Maintain the join order</span>
|
||||
</span><span id="L-278"><a href="#L-278"><span class="linenos">278</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">):</span>
|
||||
</span><span id="L-279"><a href="#L-279"><span class="linenos">279</span></a> <span class="n">position</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
</span><span id="L-280"><a href="#L-280"><span class="linenos">280</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-281"><a href="#L-281"><span class="linenos">281</span></a> <span class="n">position</span> <span class="o">=</span> <span class="n">outer_joins</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
|
||||
</span><span id="L-282"><a href="#L-282"><span class="linenos">282</span></a> <span class="n">outer_joins</span><span class="p">[</span><span class="n">position</span><span class="p">:</span><span class="n">position</span><span class="p">]</span> <span class="o">=</span> <span class="n">new_joins</span>
|
||||
</span><span id="L-283"><a href="#L-283"><span class="linenos">283</span></a>
|
||||
</span><span id="L-284"><a href="#L-284"><span class="linenos">284</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">outer_joins</span><span class="p">)</span>
|
||||
</span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a> <span class="n">new_joins</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a>
|
||||
</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="p">:</span>
|
||||
</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a> <span class="n">new_joins</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">add_source</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">])</span>
|
||||
</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a>
|
||||
</span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a> <span class="k">if</span> <span class="n">new_joins</span><span class="p">:</span>
|
||||
</span><span id="L-277"><a href="#L-277"><span class="linenos">277</span></a> <span class="n">outer_joins</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-278"><a href="#L-278"><span class="linenos">278</span></a>
|
||||
</span><span id="L-279"><a href="#L-279"><span class="linenos">279</span></a> <span class="c1"># Maintain the join order</span>
|
||||
</span><span id="L-280"><a href="#L-280"><span class="linenos">280</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">):</span>
|
||||
</span><span id="L-281"><a href="#L-281"><span class="linenos">281</span></a> <span class="n">position</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
</span><span id="L-282"><a href="#L-282"><span class="linenos">282</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-283"><a href="#L-283"><span class="linenos">283</span></a> <span class="n">position</span> <span class="o">=</span> <span class="n">outer_joins</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
|
||||
</span><span id="L-284"><a href="#L-284"><span class="linenos">284</span></a> <span class="n">outer_joins</span><span class="p">[</span><span class="n">position</span><span class="p">:</span><span class="n">position</span><span class="p">]</span> <span class="o">=</span> <span class="n">new_joins</span>
|
||||
</span><span id="L-285"><a href="#L-285"><span class="linenos">285</span></a>
|
||||
</span><span id="L-286"><a href="#L-286"><span class="linenos">286</span></a>
|
||||
</span><span id="L-287"><a href="#L-287"><span class="linenos">287</span></a><span class="k">def</span> <span class="nf">_merge_expressions</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-288"><a href="#L-288"><span class="linenos">288</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a><span class="sd"> Merge projections of inner query into outer query.</span>
|
||||
</span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a>
|
||||
</span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a><span class="sd"> alias (str)</span>
|
||||
</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-296"><a href="#L-296"><span class="linenos">296</span></a> <span class="c1"># Collect all columns that reference the alias of the inner query</span>
|
||||
</span><span id="L-297"><a href="#L-297"><span class="linenos">297</span></a> <span class="n">outer_columns</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">list</span><span class="p">)</span>
|
||||
</span><span id="L-298"><a href="#L-298"><span class="linenos">298</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-299"><a href="#L-299"><span class="linenos">299</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-300"><a href="#L-300"><span class="linenos">300</span></a> <span class="n">outer_columns</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">column</span><span class="p">)</span>
|
||||
</span><span id="L-301"><a href="#L-301"><span class="linenos">301</span></a>
|
||||
</span><span id="L-302"><a href="#L-302"><span class="linenos">302</span></a> <span class="c1"># Replace columns with the projection expression in the inner query</span>
|
||||
</span><span id="L-303"><a href="#L-303"><span class="linenos">303</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-304"><a href="#L-304"><span class="linenos">304</span></a> <span class="n">projection_name</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-305"><a href="#L-305"><span class="linenos">305</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">projection_name</span><span class="p">:</span>
|
||||
</span><span id="L-306"><a href="#L-306"><span class="linenos">306</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-307"><a href="#L-307"><span class="linenos">307</span></a> <span class="n">columns_to_replace</span> <span class="o">=</span> <span class="n">outer_columns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">projection_name</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-308"><a href="#L-308"><span class="linenos">308</span></a>
|
||||
</span><span id="L-309"><a href="#L-309"><span class="linenos">309</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">unalias</span><span class="p">()</span>
|
||||
</span><span id="L-310"><a href="#L-310"><span class="linenos">310</span></a> <span class="n">must_wrap_expression</span> <span class="o">=</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">SAFE_TO_REPLACE_UNWRAPPED</span><span class="p">)</span>
|
||||
</span><span id="L-311"><a href="#L-311"><span class="linenos">311</span></a>
|
||||
</span><span id="L-312"><a href="#L-312"><span class="linenos">312</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns_to_replace</span><span class="p">:</span>
|
||||
</span><span id="L-313"><a href="#L-313"><span class="linenos">313</span></a> <span class="c1"># Ensures we don't alter the intended operator precedence if there's additional</span>
|
||||
</span><span id="L-314"><a href="#L-314"><span class="linenos">314</span></a> <span class="c1"># context surrounding the outer expression (i.e. it's not a simple projection).</span>
|
||||
</span><span id="L-315"><a href="#L-315"><span class="linenos">315</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Unary</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">))</span> <span class="ow">and</span> <span class="n">must_wrap_expression</span><span class="p">:</span>
|
||||
</span><span id="L-316"><a href="#L-316"><span class="linenos">316</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">paren</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-317"><a href="#L-317"><span class="linenos">317</span></a>
|
||||
</span><span id="L-318"><a href="#L-318"><span class="linenos">318</span></a> <span class="n">column</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">())</span>
|
||||
</span><span id="L-286"><a href="#L-286"><span class="linenos">286</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">outer_joins</span><span class="p">)</span>
|
||||
</span><span id="L-287"><a href="#L-287"><span class="linenos">287</span></a>
|
||||
</span><span id="L-288"><a href="#L-288"><span class="linenos">288</span></a>
|
||||
</span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a><span class="k">def</span> <span class="nf">_merge_expressions</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a><span class="sd"> Merge projections of inner query into outer query.</span>
|
||||
</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a>
|
||||
</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-296"><a href="#L-296"><span class="linenos">296</span></a><span class="sd"> alias (str)</span>
|
||||
</span><span id="L-297"><a href="#L-297"><span class="linenos">297</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-298"><a href="#L-298"><span class="linenos">298</span></a> <span class="c1"># Collect all columns that reference the alias of the inner query</span>
|
||||
</span><span id="L-299"><a href="#L-299"><span class="linenos">299</span></a> <span class="n">outer_columns</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">list</span><span class="p">)</span>
|
||||
</span><span id="L-300"><a href="#L-300"><span class="linenos">300</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-301"><a href="#L-301"><span class="linenos">301</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-302"><a href="#L-302"><span class="linenos">302</span></a> <span class="n">outer_columns</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">column</span><span class="p">)</span>
|
||||
</span><span id="L-303"><a href="#L-303"><span class="linenos">303</span></a>
|
||||
</span><span id="L-304"><a href="#L-304"><span class="linenos">304</span></a> <span class="c1"># Replace columns with the projection expression in the inner query</span>
|
||||
</span><span id="L-305"><a href="#L-305"><span class="linenos">305</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-306"><a href="#L-306"><span class="linenos">306</span></a> <span class="n">projection_name</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-307"><a href="#L-307"><span class="linenos">307</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">projection_name</span><span class="p">:</span>
|
||||
</span><span id="L-308"><a href="#L-308"><span class="linenos">308</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-309"><a href="#L-309"><span class="linenos">309</span></a> <span class="n">columns_to_replace</span> <span class="o">=</span> <span class="n">outer_columns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">projection_name</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-310"><a href="#L-310"><span class="linenos">310</span></a>
|
||||
</span><span id="L-311"><a href="#L-311"><span class="linenos">311</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">unalias</span><span class="p">()</span>
|
||||
</span><span id="L-312"><a href="#L-312"><span class="linenos">312</span></a> <span class="n">must_wrap_expression</span> <span class="o">=</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">SAFE_TO_REPLACE_UNWRAPPED</span><span class="p">)</span>
|
||||
</span><span id="L-313"><a href="#L-313"><span class="linenos">313</span></a>
|
||||
</span><span id="L-314"><a href="#L-314"><span class="linenos">314</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns_to_replace</span><span class="p">:</span>
|
||||
</span><span id="L-315"><a href="#L-315"><span class="linenos">315</span></a> <span class="c1"># Ensures we don't alter the intended operator precedence if there's additional</span>
|
||||
</span><span id="L-316"><a href="#L-316"><span class="linenos">316</span></a> <span class="c1"># context surrounding the outer expression (i.e. it's not a simple projection).</span>
|
||||
</span><span id="L-317"><a href="#L-317"><span class="linenos">317</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Unary</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">))</span> <span class="ow">and</span> <span class="n">must_wrap_expression</span><span class="p">:</span>
|
||||
</span><span id="L-318"><a href="#L-318"><span class="linenos">318</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">paren</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-319"><a href="#L-319"><span class="linenos">319</span></a>
|
||||
</span><span id="L-320"><a href="#L-320"><span class="linenos">320</span></a>
|
||||
</span><span id="L-321"><a href="#L-321"><span class="linenos">321</span></a><span class="k">def</span> <span class="nf">_merge_where</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">):</span>
|
||||
</span><span id="L-322"><a href="#L-322"><span class="linenos">322</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-323"><a href="#L-323"><span class="linenos">323</span></a><span class="sd"> Merge WHERE clause of inner query into outer query.</span>
|
||||
</span><span id="L-324"><a href="#L-324"><span class="linenos">324</span></a>
|
||||
</span><span id="L-325"><a href="#L-325"><span class="linenos">325</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-326"><a href="#L-326"><span class="linenos">326</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-327"><a href="#L-327"><span class="linenos">327</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-328"><a href="#L-328"><span class="linenos">328</span></a><span class="sd"> from_or_join (exp.From|exp.Join)</span>
|
||||
</span><span id="L-329"><a href="#L-329"><span class="linenos">329</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-330"><a href="#L-330"><span class="linenos">330</span></a> <span class="n">where</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">)</span>
|
||||
</span><span id="L-331"><a href="#L-331"><span class="linenos">331</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">where</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">:</span>
|
||||
</span><span id="L-332"><a href="#L-332"><span class="linenos">332</span></a> <span class="k">return</span>
|
||||
</span><span id="L-333"><a href="#L-333"><span class="linenos">333</span></a>
|
||||
</span><span id="L-334"><a href="#L-334"><span class="linenos">334</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-320"><a href="#L-320"><span class="linenos">320</span></a> <span class="n">column</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">())</span>
|
||||
</span><span id="L-321"><a href="#L-321"><span class="linenos">321</span></a>
|
||||
</span><span id="L-322"><a href="#L-322"><span class="linenos">322</span></a>
|
||||
</span><span id="L-323"><a href="#L-323"><span class="linenos">323</span></a><span class="k">def</span> <span class="nf">_merge_where</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">):</span>
|
||||
</span><span id="L-324"><a href="#L-324"><span class="linenos">324</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-325"><a href="#L-325"><span class="linenos">325</span></a><span class="sd"> Merge WHERE clause of inner query into outer query.</span>
|
||||
</span><span id="L-326"><a href="#L-326"><span class="linenos">326</span></a>
|
||||
</span><span id="L-327"><a href="#L-327"><span class="linenos">327</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-328"><a href="#L-328"><span class="linenos">328</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-329"><a href="#L-329"><span class="linenos">329</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-330"><a href="#L-330"><span class="linenos">330</span></a><span class="sd"> from_or_join (exp.From|exp.Join)</span>
|
||||
</span><span id="L-331"><a href="#L-331"><span class="linenos">331</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-332"><a href="#L-332"><span class="linenos">332</span></a> <span class="n">where</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">)</span>
|
||||
</span><span id="L-333"><a href="#L-333"><span class="linenos">333</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">where</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">:</span>
|
||||
</span><span id="L-334"><a href="#L-334"><span class="linenos">334</span></a> <span class="k">return</span>
|
||||
</span><span id="L-335"><a href="#L-335"><span class="linenos">335</span></a>
|
||||
</span><span id="L-336"><a href="#L-336"><span class="linenos">336</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-337"><a href="#L-337"><span class="linenos">337</span></a> <span class="c1"># Merge predicates from an outer join to the ON clause</span>
|
||||
</span><span id="L-338"><a href="#L-338"><span class="linenos">338</span></a> <span class="c1"># if it only has columns that are already joined</span>
|
||||
</span><span id="L-339"><a href="#L-339"><span class="linenos">339</span></a> <span class="n">from_</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">)</span>
|
||||
</span><span id="L-340"><a href="#L-340"><span class="linenos">340</span></a> <span class="n">sources</span> <span class="o">=</span> <span class="p">{</span><span class="n">from_</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">}</span> <span class="k">if</span> <span class="n">from_</span> <span class="k">else</span> <span class="p">{}</span>
|
||||
</span><span id="L-341"><a href="#L-341"><span class="linenos">341</span></a>
|
||||
</span><span id="L-342"><a href="#L-342"><span class="linenos">342</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"joins"</span><span class="p">]:</span>
|
||||
</span><span id="L-343"><a href="#L-343"><span class="linenos">343</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-344"><a href="#L-344"><span class="linenos">344</span></a> <span class="n">sources</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
|
||||
</span><span id="L-345"><a href="#L-345"><span class="linenos">345</span></a> <span class="k">if</span> <span class="n">source</span> <span class="o">==</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span>
|
||||
</span><span id="L-346"><a href="#L-346"><span class="linenos">346</span></a> <span class="k">break</span>
|
||||
</span><span id="L-347"><a href="#L-347"><span class="linenos">347</span></a>
|
||||
</span><span id="L-348"><a href="#L-348"><span class="linenos">348</span></a> <span class="k">if</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="o"><=</span> <span class="n">sources</span><span class="p">:</span>
|
||||
</span><span id="L-349"><a href="#L-349"><span class="linenos">349</span></a> <span class="n">from_or_join</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-350"><a href="#L-350"><span class="linenos">350</span></a> <span class="n">from_or_join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"on"</span><span class="p">,</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">))</span>
|
||||
</span><span id="L-351"><a href="#L-351"><span class="linenos">351</span></a> <span class="k">return</span>
|
||||
</span><span id="L-352"><a href="#L-352"><span class="linenos">352</span></a>
|
||||
</span><span id="L-353"><a href="#L-353"><span class="linenos">353</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-336"><a href="#L-336"><span class="linenos">336</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-337"><a href="#L-337"><span class="linenos">337</span></a>
|
||||
</span><span id="L-338"><a href="#L-338"><span class="linenos">338</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-339"><a href="#L-339"><span class="linenos">339</span></a> <span class="c1"># Merge predicates from an outer join to the ON clause</span>
|
||||
</span><span id="L-340"><a href="#L-340"><span class="linenos">340</span></a> <span class="c1"># if it only has columns that are already joined</span>
|
||||
</span><span id="L-341"><a href="#L-341"><span class="linenos">341</span></a> <span class="n">from_</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">)</span>
|
||||
</span><span id="L-342"><a href="#L-342"><span class="linenos">342</span></a> <span class="n">sources</span> <span class="o">=</span> <span class="p">{</span><span class="n">from_</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">}</span> <span class="k">if</span> <span class="n">from_</span> <span class="k">else</span> <span class="p">{}</span>
|
||||
</span><span id="L-343"><a href="#L-343"><span class="linenos">343</span></a>
|
||||
</span><span id="L-344"><a href="#L-344"><span class="linenos">344</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"joins"</span><span class="p">]:</span>
|
||||
</span><span id="L-345"><a href="#L-345"><span class="linenos">345</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-346"><a href="#L-346"><span class="linenos">346</span></a> <span class="n">sources</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
|
||||
</span><span id="L-347"><a href="#L-347"><span class="linenos">347</span></a> <span class="k">if</span> <span class="n">source</span> <span class="o">==</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span>
|
||||
</span><span id="L-348"><a href="#L-348"><span class="linenos">348</span></a> <span class="k">break</span>
|
||||
</span><span id="L-349"><a href="#L-349"><span class="linenos">349</span></a>
|
||||
</span><span id="L-350"><a href="#L-350"><span class="linenos">350</span></a> <span class="k">if</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="o"><=</span> <span class="n">sources</span><span class="p">:</span>
|
||||
</span><span id="L-351"><a href="#L-351"><span class="linenos">351</span></a> <span class="n">from_or_join</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-352"><a href="#L-352"><span class="linenos">352</span></a> <span class="n">from_or_join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"on"</span><span class="p">,</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">))</span>
|
||||
</span><span id="L-353"><a href="#L-353"><span class="linenos">353</span></a> <span class="k">return</span>
|
||||
</span><span id="L-354"><a href="#L-354"><span class="linenos">354</span></a>
|
||||
</span><span id="L-355"><a href="#L-355"><span class="linenos">355</span></a>
|
||||
</span><span id="L-356"><a href="#L-356"><span class="linenos">356</span></a><span class="k">def</span> <span class="nf">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-357"><a href="#L-357"><span class="linenos">357</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-358"><a href="#L-358"><span class="linenos">358</span></a><span class="sd"> Merge ORDER clause of inner query into outer query.</span>
|
||||
</span><span id="L-359"><a href="#L-359"><span class="linenos">359</span></a>
|
||||
</span><span id="L-360"><a href="#L-360"><span class="linenos">360</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-361"><a href="#L-361"><span class="linenos">361</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-362"><a href="#L-362"><span class="linenos">362</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-363"><a href="#L-363"><span class="linenos">363</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-364"><a href="#L-364"><span class="linenos">364</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-365"><a href="#L-365"><span class="linenos">365</span></a> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-366"><a href="#L-366"><span class="linenos">366</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="p">[</span><span class="s2">"group"</span><span class="p">,</span> <span class="s2">"distinct"</span><span class="p">,</span> <span class="s2">"having"</span><span class="p">,</span> <span class="s2">"order"</span><span class="p">]</span>
|
||||
</span><span id="L-367"><a href="#L-367"><span class="linenos">367</span></a> <span class="p">)</span>
|
||||
</span><span id="L-368"><a href="#L-368"><span class="linenos">368</span></a> <span class="ow">or</span> <span class="nb">len</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span> <span class="o">!=</span> <span class="mi">1</span>
|
||||
</span><span id="L-369"><a href="#L-369"><span class="linenos">369</span></a> <span class="ow">or</span> <span class="nb">any</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">)</span> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span>
|
||||
</span><span id="L-370"><a href="#L-370"><span class="linenos">370</span></a> <span class="p">):</span>
|
||||
</span><span id="L-371"><a href="#L-371"><span class="linenos">371</span></a> <span class="k">return</span>
|
||||
</span><span id="L-372"><a href="#L-372"><span class="linenos">372</span></a>
|
||||
</span><span id="L-373"><a href="#L-373"><span class="linenos">373</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"order"</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"order"</span><span class="p">))</span>
|
||||
</span><span id="L-355"><a href="#L-355"><span class="linenos">355</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-356"><a href="#L-356"><span class="linenos">356</span></a>
|
||||
</span><span id="L-357"><a href="#L-357"><span class="linenos">357</span></a>
|
||||
</span><span id="L-358"><a href="#L-358"><span class="linenos">358</span></a><span class="k">def</span> <span class="nf">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-359"><a href="#L-359"><span class="linenos">359</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-360"><a href="#L-360"><span class="linenos">360</span></a><span class="sd"> Merge ORDER clause of inner query into outer query.</span>
|
||||
</span><span id="L-361"><a href="#L-361"><span class="linenos">361</span></a>
|
||||
</span><span id="L-362"><a href="#L-362"><span class="linenos">362</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-363"><a href="#L-363"><span class="linenos">363</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-364"><a href="#L-364"><span class="linenos">364</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-365"><a href="#L-365"><span class="linenos">365</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-366"><a href="#L-366"><span class="linenos">366</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-367"><a href="#L-367"><span class="linenos">367</span></a> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-368"><a href="#L-368"><span class="linenos">368</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="p">[</span><span class="s2">"group"</span><span class="p">,</span> <span class="s2">"distinct"</span><span class="p">,</span> <span class="s2">"having"</span><span class="p">,</span> <span class="s2">"order"</span><span class="p">]</span>
|
||||
</span><span id="L-369"><a href="#L-369"><span class="linenos">369</span></a> <span class="p">)</span>
|
||||
</span><span id="L-370"><a href="#L-370"><span class="linenos">370</span></a> <span class="ow">or</span> <span class="nb">len</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span> <span class="o">!=</span> <span class="mi">1</span>
|
||||
</span><span id="L-371"><a href="#L-371"><span class="linenos">371</span></a> <span class="ow">or</span> <span class="nb">any</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">)</span> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span>
|
||||
</span><span id="L-372"><a href="#L-372"><span class="linenos">372</span></a> <span class="p">):</span>
|
||||
</span><span id="L-373"><a href="#L-373"><span class="linenos">373</span></a> <span class="k">return</span>
|
||||
</span><span id="L-374"><a href="#L-374"><span class="linenos">374</span></a>
|
||||
</span><span id="L-375"><a href="#L-375"><span class="linenos">375</span></a>
|
||||
</span><span id="L-376"><a href="#L-376"><span class="linenos">376</span></a><span class="k">def</span> <span class="nf">_merge_hints</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-377"><a href="#L-377"><span class="linenos">377</span></a> <span class="n">inner_scope_hint</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">)</span>
|
||||
</span><span id="L-378"><a href="#L-378"><span class="linenos">378</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">inner_scope_hint</span><span class="p">:</span>
|
||||
</span><span id="L-379"><a href="#L-379"><span class="linenos">379</span></a> <span class="k">return</span>
|
||||
</span><span id="L-380"><a href="#L-380"><span class="linenos">380</span></a> <span class="n">outer_scope_hint</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">)</span>
|
||||
</span><span id="L-381"><a href="#L-381"><span class="linenos">381</span></a> <span class="k">if</span> <span class="n">outer_scope_hint</span><span class="p">:</span>
|
||||
</span><span id="L-382"><a href="#L-382"><span class="linenos">382</span></a> <span class="k">for</span> <span class="n">hint_expression</span> <span class="ow">in</span> <span class="n">inner_scope_hint</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-383"><a href="#L-383"><span class="linenos">383</span></a> <span class="n">outer_scope_hint</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"expressions"</span><span class="p">,</span> <span class="n">hint_expression</span><span class="p">)</span>
|
||||
</span><span id="L-384"><a href="#L-384"><span class="linenos">384</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-385"><a href="#L-385"><span class="linenos">385</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">,</span> <span class="n">inner_scope_hint</span><span class="p">)</span>
|
||||
</span><span id="L-386"><a href="#L-386"><span class="linenos">386</span></a>
|
||||
</span><span id="L-387"><a href="#L-387"><span class="linenos">387</span></a>
|
||||
</span><span id="L-388"><a href="#L-388"><span class="linenos">388</span></a><span class="k">def</span> <span class="nf">_pop_cte</span><span class="p">(</span><span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-389"><a href="#L-389"><span class="linenos">389</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-390"><a href="#L-390"><span class="linenos">390</span></a><span class="sd"> Remove CTE from the AST.</span>
|
||||
</span><span id="L-391"><a href="#L-391"><span class="linenos">391</span></a>
|
||||
</span><span id="L-392"><a href="#L-392"><span class="linenos">392</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-393"><a href="#L-393"><span class="linenos">393</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-394"><a href="#L-394"><span class="linenos">394</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-395"><a href="#L-395"><span class="linenos">395</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-396"><a href="#L-396"><span class="linenos">396</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">cte</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-397"><a href="#L-397"><span class="linenos">397</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">with_</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-398"><a href="#L-398"><span class="linenos">398</span></a> <span class="n">with_</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-399"><a href="#L-399"><span class="linenos">399</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-400"><a href="#L-400"><span class="linenos">400</span></a> <span class="n">cte</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-375"><a href="#L-375"><span class="linenos">375</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"order"</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"order"</span><span class="p">))</span>
|
||||
</span><span id="L-376"><a href="#L-376"><span class="linenos">376</span></a>
|
||||
</span><span id="L-377"><a href="#L-377"><span class="linenos">377</span></a>
|
||||
</span><span id="L-378"><a href="#L-378"><span class="linenos">378</span></a><span class="k">def</span> <span class="nf">_merge_hints</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-379"><a href="#L-379"><span class="linenos">379</span></a> <span class="n">inner_scope_hint</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">)</span>
|
||||
</span><span id="L-380"><a href="#L-380"><span class="linenos">380</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">inner_scope_hint</span><span class="p">:</span>
|
||||
</span><span id="L-381"><a href="#L-381"><span class="linenos">381</span></a> <span class="k">return</span>
|
||||
</span><span id="L-382"><a href="#L-382"><span class="linenos">382</span></a> <span class="n">outer_scope_hint</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">)</span>
|
||||
</span><span id="L-383"><a href="#L-383"><span class="linenos">383</span></a> <span class="k">if</span> <span class="n">outer_scope_hint</span><span class="p">:</span>
|
||||
</span><span id="L-384"><a href="#L-384"><span class="linenos">384</span></a> <span class="k">for</span> <span class="n">hint_expression</span> <span class="ow">in</span> <span class="n">inner_scope_hint</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-385"><a href="#L-385"><span class="linenos">385</span></a> <span class="n">outer_scope_hint</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"expressions"</span><span class="p">,</span> <span class="n">hint_expression</span><span class="p">)</span>
|
||||
</span><span id="L-386"><a href="#L-386"><span class="linenos">386</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-387"><a href="#L-387"><span class="linenos">387</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">,</span> <span class="n">inner_scope_hint</span><span class="p">)</span>
|
||||
</span><span id="L-388"><a href="#L-388"><span class="linenos">388</span></a>
|
||||
</span><span id="L-389"><a href="#L-389"><span class="linenos">389</span></a>
|
||||
</span><span id="L-390"><a href="#L-390"><span class="linenos">390</span></a><span class="k">def</span> <span class="nf">_pop_cte</span><span class="p">(</span><span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-391"><a href="#L-391"><span class="linenos">391</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-392"><a href="#L-392"><span class="linenos">392</span></a><span class="sd"> Remove CTE from the AST.</span>
|
||||
</span><span id="L-393"><a href="#L-393"><span class="linenos">393</span></a>
|
||||
</span><span id="L-394"><a href="#L-394"><span class="linenos">394</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-395"><a href="#L-395"><span class="linenos">395</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-396"><a href="#L-396"><span class="linenos">396</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-397"><a href="#L-397"><span class="linenos">397</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-398"><a href="#L-398"><span class="linenos">398</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">cte</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-399"><a href="#L-399"><span class="linenos">399</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">with_</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-400"><a href="#L-400"><span class="linenos">400</span></a> <span class="n">with_</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-401"><a href="#L-401"><span class="linenos">401</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-402"><a href="#L-402"><span class="linenos">402</span></a> <span class="n">cte</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -566,7 +568,7 @@ queries if it would result in multiple table selects in a single query:</p>
|
|||
<div class="attr variable">
|
||||
<span class="name">UNMERGABLE_ARGS</span> =
|
||||
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{'into', 'windows', 'cluster', 'match', 'distinct', 'group', 'with', 'distribute', 'locks', 'laterals', 'sample', 'having', 'limit', 'sort', 'pivots', 'kind', 'qualify', 'settings', 'offset', 'format'}</span>
|
||||
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{'kind', 'offset', 'settings', 'locks', 'match', 'with', 'into', 'distribute', 'cluster', 'format', 'laterals', 'limit', 'group', 'sample', 'sort', 'windows', 'pivots', 'having', 'distinct', 'qualify'}</span>
|
||||
|
||||
|
||||
</div>
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -125,63 +125,66 @@
|
|||
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="k">if</span> <span class="n">remove_unused_selections</span><span class="p">:</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="n">_remove_unused_selections</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">parent_selections</span><span class="p">,</span> <span class="n">schema</span><span class="p">)</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="c1"># Group columns by source name</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="n">selects</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">set</span><span class="p">)</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">table</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="n">col_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="n">selects</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="c1"># Push the selected columns down to the next scope</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="p">(</span><span class="n">_</span><span class="p">,</span> <span class="n">source</span><span class="p">)</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="n">selects</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="ow">or</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">source</span><span class="p">]</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">is_star</span><span class="p">:</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="c1"># Group columns by source name</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="n">selects</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">set</span><span class="p">)</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">table</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">col_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">selects</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="c1"># Push the selected columns down to the next scope</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="p">(</span><span class="n">_</span><span class="p">,</span> <span class="n">source</span><span class="p">)</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="n">selects</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="ow">or</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">source</span><span class="p">]</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a><span class="k">def</span> <span class="nf">_remove_unused_selections</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">parent_selections</span><span class="p">,</span> <span class="n">schema</span><span class="p">):</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">order</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"order"</span><span class="p">)</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">if</span> <span class="n">order</span><span class="p">:</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="c1"># Assume columns without a qualified table are references to output columns</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="n">order_refs</span> <span class="o">=</span> <span class="p">{</span><span class="n">c</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">order</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">c</span><span class="o">.</span><span class="n">table</span><span class="p">}</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="n">order_refs</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">new_selections</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="n">removed</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="n">star</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">for</span> <span class="n">selection</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">selection</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a><span class="k">def</span> <span class="nf">_remove_unused_selections</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">parent_selections</span><span class="p">,</span> <span class="n">schema</span><span class="p">):</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="n">order</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"order"</span><span class="p">)</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">if</span> <span class="n">order</span><span class="p">:</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="c1"># Assume columns without a qualified table are references to output columns</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="n">order_refs</span> <span class="o">=</span> <span class="p">{</span><span class="n">c</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">order</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">c</span><span class="o">.</span><span class="n">table</span><span class="p">}</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="n">order_refs</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">new_selections</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="n">removed</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">star</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="k">if</span> <span class="n">SELECT_ALL</span> <span class="ow">in</span> <span class="n">parent_selections</span> <span class="ow">or</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">parent_selections</span> <span class="ow">or</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">order_refs</span><span class="p">:</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">new_selections</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">selection</span><span class="p">)</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="k">if</span> <span class="n">selection</span><span class="o">.</span><span class="n">is_star</span><span class="p">:</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">star</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">removed</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="k">if</span> <span class="n">star</span><span class="p">:</span>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="n">resolver</span> <span class="o">=</span> <span class="n">Resolver</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">schema</span><span class="p">)</span>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="n">names</span> <span class="o">=</span> <span class="p">{</span><span class="n">s</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">new_selections</span><span class="p">}</span>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">parent_selections</span><span class="p">):</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">names</span><span class="p">:</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">new_selections</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="n">resolver</span><span class="o">.</span><span class="n">get_table</span><span class="p">(</span><span class="n">name</span><span class="p">)),</span> <span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="p">)</span>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="c1"># If there are no remaining selections, just select a single constant</span>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">new_selections</span><span class="p">:</span>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">new_selections</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">DEFAULT_SELECTION</span><span class="p">())</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="o">*</span><span class="n">new_selections</span><span class="p">,</span> <span class="n">append</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="k">if</span> <span class="n">removed</span><span class="p">:</span>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">clear_cache</span><span class="p">()</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="k">for</span> <span class="n">selection</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">selection</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="k">if</span> <span class="n">SELECT_ALL</span> <span class="ow">in</span> <span class="n">parent_selections</span> <span class="ow">or</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">parent_selections</span> <span class="ow">or</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">order_refs</span><span class="p">:</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">new_selections</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">selection</span><span class="p">)</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">if</span> <span class="n">selection</span><span class="o">.</span><span class="n">is_star</span><span class="p">:</span>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">star</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="n">removed</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="k">if</span> <span class="n">star</span><span class="p">:</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="n">resolver</span> <span class="o">=</span> <span class="n">Resolver</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">schema</span><span class="p">)</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">names</span> <span class="o">=</span> <span class="p">{</span><span class="n">s</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">new_selections</span><span class="p">}</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">parent_selections</span><span class="p">):</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">names</span><span class="p">:</span>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="n">new_selections</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="n">resolver</span><span class="o">.</span><span class="n">get_table</span><span class="p">(</span><span class="n">name</span><span class="p">)),</span> <span class="n">name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="p">)</span>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="c1"># If there are no remaining selections, just select a single constant</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">new_selections</span><span class="p">:</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">new_selections</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">DEFAULT_SELECTION</span><span class="p">())</span>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="o">*</span><span class="n">new_selections</span><span class="p">,</span> <span class="n">append</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="k">if</span> <span class="n">removed</span><span class="p">:</span>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">clear_cache</span><span class="p">()</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -276,20 +279,23 @@
|
|||
</span><span id="pushdown_projections-62"><a href="#pushdown_projections-62"><span class="linenos">62</span></a> <span class="k">if</span> <span class="n">remove_unused_selections</span><span class="p">:</span>
|
||||
</span><span id="pushdown_projections-63"><a href="#pushdown_projections-63"><span class="linenos">63</span></a> <span class="n">_remove_unused_selections</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">parent_selections</span><span class="p">,</span> <span class="n">schema</span><span class="p">)</span>
|
||||
</span><span id="pushdown_projections-64"><a href="#pushdown_projections-64"><span class="linenos">64</span></a>
|
||||
</span><span id="pushdown_projections-65"><a href="#pushdown_projections-65"><span class="linenos">65</span></a> <span class="c1"># Group columns by source name</span>
|
||||
</span><span id="pushdown_projections-66"><a href="#pushdown_projections-66"><span class="linenos">66</span></a> <span class="n">selects</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">set</span><span class="p">)</span>
|
||||
</span><span id="pushdown_projections-67"><a href="#pushdown_projections-67"><span class="linenos">67</span></a> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="pushdown_projections-68"><a href="#pushdown_projections-68"><span class="linenos">68</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">table</span>
|
||||
</span><span id="pushdown_projections-69"><a href="#pushdown_projections-69"><span class="linenos">69</span></a> <span class="n">col_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="pushdown_projections-70"><a href="#pushdown_projections-70"><span class="linenos">70</span></a> <span class="n">selects</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span>
|
||||
</span><span id="pushdown_projections-71"><a href="#pushdown_projections-71"><span class="linenos">71</span></a>
|
||||
</span><span id="pushdown_projections-72"><a href="#pushdown_projections-72"><span class="linenos">72</span></a> <span class="c1"># Push the selected columns down to the next scope</span>
|
||||
</span><span id="pushdown_projections-73"><a href="#pushdown_projections-73"><span class="linenos">73</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="p">(</span><span class="n">_</span><span class="p">,</span> <span class="n">source</span><span class="p">)</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="pushdown_projections-74"><a href="#pushdown_projections-74"><span class="linenos">74</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span>
|
||||
</span><span id="pushdown_projections-75"><a href="#pushdown_projections-75"><span class="linenos">75</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="n">selects</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="ow">or</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="pushdown_projections-76"><a href="#pushdown_projections-76"><span class="linenos">76</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">source</span><span class="p">]</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="pushdown_projections-77"><a href="#pushdown_projections-77"><span class="linenos">77</span></a>
|
||||
</span><span id="pushdown_projections-78"><a href="#pushdown_projections-78"><span class="linenos">78</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="pushdown_projections-65"><a href="#pushdown_projections-65"><span class="linenos">65</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">is_star</span><span class="p">:</span>
|
||||
</span><span id="pushdown_projections-66"><a href="#pushdown_projections-66"><span class="linenos">66</span></a> <span class="k">continue</span>
|
||||
</span><span id="pushdown_projections-67"><a href="#pushdown_projections-67"><span class="linenos">67</span></a>
|
||||
</span><span id="pushdown_projections-68"><a href="#pushdown_projections-68"><span class="linenos">68</span></a> <span class="c1"># Group columns by source name</span>
|
||||
</span><span id="pushdown_projections-69"><a href="#pushdown_projections-69"><span class="linenos">69</span></a> <span class="n">selects</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">set</span><span class="p">)</span>
|
||||
</span><span id="pushdown_projections-70"><a href="#pushdown_projections-70"><span class="linenos">70</span></a> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="pushdown_projections-71"><a href="#pushdown_projections-71"><span class="linenos">71</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">table</span>
|
||||
</span><span id="pushdown_projections-72"><a href="#pushdown_projections-72"><span class="linenos">72</span></a> <span class="n">col_name</span> <span class="o">=</span> <span class="n">col</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="pushdown_projections-73"><a href="#pushdown_projections-73"><span class="linenos">73</span></a> <span class="n">selects</span><span class="p">[</span><span class="n">table_name</span><span class="p">]</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">col_name</span><span class="p">)</span>
|
||||
</span><span id="pushdown_projections-74"><a href="#pushdown_projections-74"><span class="linenos">74</span></a>
|
||||
</span><span id="pushdown_projections-75"><a href="#pushdown_projections-75"><span class="linenos">75</span></a> <span class="c1"># Push the selected columns down to the next scope</span>
|
||||
</span><span id="pushdown_projections-76"><a href="#pushdown_projections-76"><span class="linenos">76</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="p">(</span><span class="n">_</span><span class="p">,</span> <span class="n">source</span><span class="p">)</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="pushdown_projections-77"><a href="#pushdown_projections-77"><span class="linenos">77</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">):</span>
|
||||
</span><span id="pushdown_projections-78"><a href="#pushdown_projections-78"><span class="linenos">78</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="n">selects</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="ow">or</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="pushdown_projections-79"><a href="#pushdown_projections-79"><span class="linenos">79</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">source</span><span class="p">]</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="pushdown_projections-80"><a href="#pushdown_projections-80"><span class="linenos">80</span></a>
|
||||
</span><span id="pushdown_projections-81"><a href="#pushdown_projections-81"><span class="linenos">81</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -356,8 +356,8 @@
|
|||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_ctes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="k">elif</span> <span class="p">(</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">)</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="ow">and</span> <span class="n">_is_subquery_scope</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">))</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="ow">and</span> <span class="n">_is_derived_table</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="p">):</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_derived_tables</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subqueryable</span><span class="p">):</span>
|
||||
|
|
@ -842,13 +842,13 @@
|
|||
</span><span id="L-610"><a href="#L-610"><span class="linenos">610</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">sources</span><span class="p">)</span>
|
||||
</span><span id="L-611"><a href="#L-611"><span class="linenos">611</span></a>
|
||||
</span><span id="L-612"><a href="#L-612"><span class="linenos">612</span></a>
|
||||
</span><span id="L-613"><a href="#L-613"><span class="linenos">613</span></a><span class="k">def</span> <span class="nf">_is_subquery_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="L-613"><a href="#L-613"><span class="linenos">613</span></a><span class="k">def</span> <span class="nf">_is_derived_table</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="L-614"><a href="#L-614"><span class="linenos">614</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-615"><a href="#L-615"><span class="linenos">615</span></a><span class="sd"> We represent (tbl1 JOIN tbl2) as a Subquery, but it's not really a new scope.</span>
|
||||
</span><span id="L-616"><a href="#L-616"><span class="linenos">616</span></a><span class="sd"> If an alias is present, it shadows all names under the Subquery, so that's an</span>
|
||||
</span><span id="L-617"><a href="#L-617"><span class="linenos">617</span></a><span class="sd"> exception to this rule.</span>
|
||||
</span><span id="L-615"><a href="#L-615"><span class="linenos">615</span></a><span class="sd"> We represent (tbl1 JOIN tbl2) as a Subquery, but it's not really a "derived table",</span>
|
||||
</span><span id="L-616"><a href="#L-616"><span class="linenos">616</span></a><span class="sd"> as it doesn't introduce a new scope. If an alias is present, it shadows all names</span>
|
||||
</span><span id="L-617"><a href="#L-617"><span class="linenos">617</span></a><span class="sd"> under the Subquery, so that's one exception to this rule.</span>
|
||||
</span><span id="L-618"><a href="#L-618"><span class="linenos">618</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-619"><a href="#L-619"><span class="linenos">619</span></a> <span class="k">return</span> <span class="nb">bool</span><span class="p">(</span><span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">unnest</span><span class="p">(),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="ow">or</span> <span class="n">expression</span><span class="o">.</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="L-619"><a href="#L-619"><span class="linenos">619</span></a> <span class="k">return</span> <span class="nb">bool</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">alias</span> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subqueryable</span><span class="p">))</span>
|
||||
</span><span id="L-620"><a href="#L-620"><span class="linenos">620</span></a>
|
||||
</span><span id="L-621"><a href="#L-621"><span class="linenos">621</span></a>
|
||||
</span><span id="L-622"><a href="#L-622"><span class="linenos">622</span></a><span class="k">def</span> <span class="nf">_traverse_tables</span><span class="p">(</span><span class="n">scope</span><span class="p">):</span>
|
||||
|
|
@ -886,94 +886,104 @@
|
|||
</span><span id="L-654"><a href="#L-654"><span class="linenos">654</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-655"><a href="#L-655"><span class="linenos">655</span></a> <span class="n">sources</span><span class="p">[</span><span class="n">source_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">expression</span>
|
||||
</span><span id="L-656"><a href="#L-656"><span class="linenos">656</span></a>
|
||||
</span><span id="L-657"><a href="#L-657"><span class="linenos">657</span></a> <span class="n">expressions</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">this</span> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[])</span>
|
||||
</span><span id="L-658"><a href="#L-658"><span class="linenos">658</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-659"><a href="#L-659"><span class="linenos">659</span></a>
|
||||
</span><span id="L-660"><a href="#L-660"><span class="linenos">660</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DerivedTable</span><span class="p">):</span>
|
||||
</span><span id="L-657"><a href="#L-657"><span class="linenos">657</span></a> <span class="c1"># Make sure to not include the joins twice</span>
|
||||
</span><span id="L-658"><a href="#L-658"><span class="linenos">658</span></a> <span class="k">if</span> <span class="n">expression</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">:</span>
|
||||
</span><span id="L-659"><a href="#L-659"><span class="linenos">659</span></a> <span class="n">expressions</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">this</span> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[])</span>
|
||||
</span><span id="L-660"><a href="#L-660"><span class="linenos">660</span></a>
|
||||
</span><span id="L-661"><a href="#L-661"><span class="linenos">661</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-662"><a href="#L-662"><span class="linenos">662</span></a>
|
||||
</span><span id="L-663"><a href="#L-663"><span class="linenos">663</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">UDTF</span><span class="p">):</span>
|
||||
</span><span id="L-664"><a href="#L-664"><span class="linenos">664</span></a> <span class="n">lateral_sources</span> <span class="o">=</span> <span class="n">sources</span>
|
||||
</span><span id="L-665"><a href="#L-665"><span class="linenos">665</span></a> <span class="n">scope_type</span> <span class="o">=</span> <span class="n">ScopeType</span><span class="o">.</span><span class="n">UDTF</span>
|
||||
</span><span id="L-666"><a href="#L-666"><span class="linenos">666</span></a> <span class="n">scopes</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">udtf_scopes</span>
|
||||
</span><span id="L-667"><a href="#L-667"><span class="linenos">667</span></a> <span class="k">elif</span> <span class="n">_is_subquery_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-668"><a href="#L-668"><span class="linenos">668</span></a> <span class="n">lateral_sources</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-669"><a href="#L-669"><span class="linenos">669</span></a> <span class="n">scope_type</span> <span class="o">=</span> <span class="n">ScopeType</span><span class="o">.</span><span class="n">DERIVED_TABLE</span>
|
||||
</span><span id="L-670"><a href="#L-670"><span class="linenos">670</span></a> <span class="n">scopes</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_table_scopes</span>
|
||||
</span><span id="L-671"><a href="#L-671"><span class="linenos">671</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-672"><a href="#L-672"><span class="linenos">672</span></a> <span class="c1"># Makes sure we check for possible sources in nested table constructs</span>
|
||||
</span><span id="L-673"><a href="#L-673"><span class="linenos">673</span></a> <span class="n">expressions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="L-663"><a href="#L-663"><span class="linenos">663</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">DerivedTable</span><span class="p">):</span>
|
||||
</span><span id="L-664"><a href="#L-664"><span class="linenos">664</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-665"><a href="#L-665"><span class="linenos">665</span></a>
|
||||
</span><span id="L-666"><a href="#L-666"><span class="linenos">666</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">UDTF</span><span class="p">):</span>
|
||||
</span><span id="L-667"><a href="#L-667"><span class="linenos">667</span></a> <span class="n">lateral_sources</span> <span class="o">=</span> <span class="n">sources</span>
|
||||
</span><span id="L-668"><a href="#L-668"><span class="linenos">668</span></a> <span class="n">scope_type</span> <span class="o">=</span> <span class="n">ScopeType</span><span class="o">.</span><span class="n">UDTF</span>
|
||||
</span><span id="L-669"><a href="#L-669"><span class="linenos">669</span></a> <span class="n">scopes</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">udtf_scopes</span>
|
||||
</span><span id="L-670"><a href="#L-670"><span class="linenos">670</span></a> <span class="k">elif</span> <span class="n">_is_derived_table</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-671"><a href="#L-671"><span class="linenos">671</span></a> <span class="n">lateral_sources</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-672"><a href="#L-672"><span class="linenos">672</span></a> <span class="n">scope_type</span> <span class="o">=</span> <span class="n">ScopeType</span><span class="o">.</span><span class="n">DERIVED_TABLE</span>
|
||||
</span><span id="L-673"><a href="#L-673"><span class="linenos">673</span></a> <span class="n">scopes</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_table_scopes</span>
|
||||
</span><span id="L-674"><a href="#L-674"><span class="linenos">674</span></a> <span class="n">expressions</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">this</span> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[])</span>
|
||||
</span><span id="L-675"><a href="#L-675"><span class="linenos">675</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-676"><a href="#L-676"><span class="linenos">676</span></a>
|
||||
</span><span id="L-677"><a href="#L-677"><span class="linenos">677</span></a> <span class="k">for</span> <span class="n">child_scope</span> <span class="ow">in</span> <span class="n">_traverse_scope</span><span class="p">(</span>
|
||||
</span><span id="L-678"><a href="#L-678"><span class="linenos">678</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">branch</span><span class="p">(</span>
|
||||
</span><span id="L-679"><a href="#L-679"><span class="linenos">679</span></a> <span class="n">expression</span><span class="p">,</span>
|
||||
</span><span id="L-680"><a href="#L-680"><span class="linenos">680</span></a> <span class="n">lateral_sources</span><span class="o">=</span><span class="n">lateral_sources</span><span class="p">,</span>
|
||||
</span><span id="L-681"><a href="#L-681"><span class="linenos">681</span></a> <span class="n">outer_column_list</span><span class="o">=</span><span class="n">expression</span><span class="o">.</span><span class="n">alias_column_names</span><span class="p">,</span>
|
||||
</span><span id="L-682"><a href="#L-682"><span class="linenos">682</span></a> <span class="n">scope_type</span><span class="o">=</span><span class="n">scope_type</span><span class="p">,</span>
|
||||
</span><span id="L-683"><a href="#L-683"><span class="linenos">683</span></a> <span class="p">)</span>
|
||||
</span><span id="L-684"><a href="#L-684"><span class="linenos">684</span></a> <span class="p">):</span>
|
||||
</span><span id="L-685"><a href="#L-685"><span class="linenos">685</span></a> <span class="k">yield</span> <span class="n">child_scope</span>
|
||||
</span><span id="L-686"><a href="#L-686"><span class="linenos">686</span></a>
|
||||
</span><span id="L-687"><a href="#L-687"><span class="linenos">687</span></a> <span class="c1"># Tables without aliases will be set as ""</span>
|
||||
</span><span id="L-688"><a href="#L-688"><span class="linenos">688</span></a> <span class="c1"># This shouldn't be a problem once qualify_columns runs, as it adds aliases on everything.</span>
|
||||
</span><span id="L-689"><a href="#L-689"><span class="linenos">689</span></a> <span class="c1"># Until then, this means that only a single, unaliased derived table is allowed (rather,</span>
|
||||
</span><span id="L-690"><a href="#L-690"><span class="linenos">690</span></a> <span class="c1"># the latest one wins.</span>
|
||||
</span><span id="L-691"><a href="#L-691"><span class="linenos">691</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="L-692"><a href="#L-692"><span class="linenos">692</span></a> <span class="n">sources</span><span class="p">[</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">child_scope</span>
|
||||
</span><span id="L-693"><a href="#L-693"><span class="linenos">693</span></a>
|
||||
</span><span id="L-694"><a href="#L-694"><span class="linenos">694</span></a> <span class="c1"># append the final child_scope yielded</span>
|
||||
</span><span id="L-695"><a href="#L-695"><span class="linenos">695</span></a> <span class="n">scopes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">child_scope</span><span class="p">)</span>
|
||||
</span><span id="L-696"><a href="#L-696"><span class="linenos">696</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">table_scopes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">child_scope</span><span class="p">)</span>
|
||||
</span><span id="L-675"><a href="#L-675"><span class="linenos">675</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-676"><a href="#L-676"><span class="linenos">676</span></a> <span class="c1"># Makes sure we check for possible sources in nested table constructs</span>
|
||||
</span><span id="L-677"><a href="#L-677"><span class="linenos">677</span></a> <span class="n">expressions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="L-678"><a href="#L-678"><span class="linenos">678</span></a> <span class="n">expressions</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">this</span> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[])</span>
|
||||
</span><span id="L-679"><a href="#L-679"><span class="linenos">679</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-680"><a href="#L-680"><span class="linenos">680</span></a>
|
||||
</span><span id="L-681"><a href="#L-681"><span class="linenos">681</span></a> <span class="k">for</span> <span class="n">child_scope</span> <span class="ow">in</span> <span class="n">_traverse_scope</span><span class="p">(</span>
|
||||
</span><span id="L-682"><a href="#L-682"><span class="linenos">682</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">branch</span><span class="p">(</span>
|
||||
</span><span id="L-683"><a href="#L-683"><span class="linenos">683</span></a> <span class="n">expression</span><span class="p">,</span>
|
||||
</span><span id="L-684"><a href="#L-684"><span class="linenos">684</span></a> <span class="n">lateral_sources</span><span class="o">=</span><span class="n">lateral_sources</span><span class="p">,</span>
|
||||
</span><span id="L-685"><a href="#L-685"><span class="linenos">685</span></a> <span class="n">outer_column_list</span><span class="o">=</span><span class="n">expression</span><span class="o">.</span><span class="n">alias_column_names</span><span class="p">,</span>
|
||||
</span><span id="L-686"><a href="#L-686"><span class="linenos">686</span></a> <span class="n">scope_type</span><span class="o">=</span><span class="n">scope_type</span><span class="p">,</span>
|
||||
</span><span id="L-687"><a href="#L-687"><span class="linenos">687</span></a> <span class="p">)</span>
|
||||
</span><span id="L-688"><a href="#L-688"><span class="linenos">688</span></a> <span class="p">):</span>
|
||||
</span><span id="L-689"><a href="#L-689"><span class="linenos">689</span></a> <span class="k">yield</span> <span class="n">child_scope</span>
|
||||
</span><span id="L-690"><a href="#L-690"><span class="linenos">690</span></a>
|
||||
</span><span id="L-691"><a href="#L-691"><span class="linenos">691</span></a> <span class="c1"># Tables without aliases will be set as ""</span>
|
||||
</span><span id="L-692"><a href="#L-692"><span class="linenos">692</span></a> <span class="c1"># This shouldn't be a problem once qualify_columns runs, as it adds aliases on everything.</span>
|
||||
</span><span id="L-693"><a href="#L-693"><span class="linenos">693</span></a> <span class="c1"># Until then, this means that only a single, unaliased derived table is allowed (rather,</span>
|
||||
</span><span id="L-694"><a href="#L-694"><span class="linenos">694</span></a> <span class="c1"># the latest one wins.</span>
|
||||
</span><span id="L-695"><a href="#L-695"><span class="linenos">695</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">alias</span>
|
||||
</span><span id="L-696"><a href="#L-696"><span class="linenos">696</span></a> <span class="n">sources</span><span class="p">[</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">child_scope</span>
|
||||
</span><span id="L-697"><a href="#L-697"><span class="linenos">697</span></a>
|
||||
</span><span id="L-698"><a href="#L-698"><span class="linenos">698</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">sources</span><span class="p">)</span>
|
||||
</span><span id="L-699"><a href="#L-699"><span class="linenos">699</span></a>
|
||||
</span><span id="L-700"><a href="#L-700"><span class="linenos">700</span></a>
|
||||
</span><span id="L-701"><a href="#L-701"><span class="linenos">701</span></a><span class="k">def</span> <span class="nf">_traverse_subqueries</span><span class="p">(</span><span class="n">scope</span><span class="p">):</span>
|
||||
</span><span id="L-702"><a href="#L-702"><span class="linenos">702</span></a> <span class="k">for</span> <span class="n">subquery</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">subqueries</span><span class="p">:</span>
|
||||
</span><span id="L-703"><a href="#L-703"><span class="linenos">703</span></a> <span class="n">top</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-704"><a href="#L-704"><span class="linenos">704</span></a> <span class="k">for</span> <span class="n">child_scope</span> <span class="ow">in</span> <span class="n">_traverse_scope</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">branch</span><span class="p">(</span><span class="n">subquery</span><span class="p">,</span> <span class="n">scope_type</span><span class="o">=</span><span class="n">ScopeType</span><span class="o">.</span><span class="n">SUBQUERY</span><span class="p">)):</span>
|
||||
</span><span id="L-705"><a href="#L-705"><span class="linenos">705</span></a> <span class="k">yield</span> <span class="n">child_scope</span>
|
||||
</span><span id="L-706"><a href="#L-706"><span class="linenos">706</span></a> <span class="n">top</span> <span class="o">=</span> <span class="n">child_scope</span>
|
||||
</span><span id="L-707"><a href="#L-707"><span class="linenos">707</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">subquery_scopes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">top</span><span class="p">)</span>
|
||||
</span><span id="L-708"><a href="#L-708"><span class="linenos">708</span></a>
|
||||
</span><span id="L-709"><a href="#L-709"><span class="linenos">709</span></a>
|
||||
</span><span id="L-710"><a href="#L-710"><span class="linenos">710</span></a><span class="k">def</span> <span class="nf">walk_in_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">bfs</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span>
|
||||
</span><span id="L-711"><a href="#L-711"><span class="linenos">711</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-712"><a href="#L-712"><span class="linenos">712</span></a><span class="sd"> Returns a generator object which visits all nodes in the syntrax tree, stopping at</span>
|
||||
</span><span id="L-713"><a href="#L-713"><span class="linenos">713</span></a><span class="sd"> nodes that start child scopes.</span>
|
||||
</span><span id="L-714"><a href="#L-714"><span class="linenos">714</span></a>
|
||||
</span><span id="L-715"><a href="#L-715"><span class="linenos">715</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-716"><a href="#L-716"><span class="linenos">716</span></a><span class="sd"> expression (exp.Expression):</span>
|
||||
</span><span id="L-717"><a href="#L-717"><span class="linenos">717</span></a><span class="sd"> bfs (bool): if set to True the BFS traversal order will be applied,</span>
|
||||
</span><span id="L-718"><a href="#L-718"><span class="linenos">718</span></a><span class="sd"> otherwise the DFS traversal will be used instead.</span>
|
||||
</span><span id="L-719"><a href="#L-719"><span class="linenos">719</span></a>
|
||||
</span><span id="L-720"><a href="#L-720"><span class="linenos">720</span></a><span class="sd"> Yields:</span>
|
||||
</span><span id="L-721"><a href="#L-721"><span class="linenos">721</span></a><span class="sd"> tuple[exp.Expression, Optional[exp.Expression], str]: node, parent, arg key</span>
|
||||
</span><span id="L-722"><a href="#L-722"><span class="linenos">722</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-723"><a href="#L-723"><span class="linenos">723</span></a> <span class="c1"># We'll use this variable to pass state into the dfs generator.</span>
|
||||
</span><span id="L-724"><a href="#L-724"><span class="linenos">724</span></a> <span class="c1"># Whenever we set it to True, we exclude a subtree from traversal.</span>
|
||||
</span><span id="L-725"><a href="#L-725"><span class="linenos">725</span></a> <span class="n">prune</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="L-726"><a href="#L-726"><span class="linenos">726</span></a>
|
||||
</span><span id="L-727"><a href="#L-727"><span class="linenos">727</span></a> <span class="k">for</span> <span class="n">node</span><span class="p">,</span> <span class="n">parent</span><span class="p">,</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">bfs</span><span class="o">=</span><span class="n">bfs</span><span class="p">,</span> <span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="o">*</span><span class="n">_</span><span class="p">:</span> <span class="n">prune</span><span class="p">):</span>
|
||||
</span><span id="L-728"><a href="#L-728"><span class="linenos">728</span></a> <span class="n">prune</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="L-729"><a href="#L-729"><span class="linenos">729</span></a>
|
||||
</span><span id="L-730"><a href="#L-730"><span class="linenos">730</span></a> <span class="k">yield</span> <span class="n">node</span><span class="p">,</span> <span class="n">parent</span><span class="p">,</span> <span class="n">key</span>
|
||||
</span><span id="L-731"><a href="#L-731"><span class="linenos">731</span></a>
|
||||
</span><span id="L-732"><a href="#L-732"><span class="linenos">732</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">is</span> <span class="n">expression</span><span class="p">:</span>
|
||||
</span><span id="L-733"><a href="#L-733"><span class="linenos">733</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-734"><a href="#L-734"><span class="linenos">734</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-735"><a href="#L-735"><span class="linenos">735</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">CTE</span><span class="p">)</span>
|
||||
</span><span id="L-736"><a href="#L-736"><span class="linenos">736</span></a> <span class="ow">or</span> <span class="p">(</span>
|
||||
</span><span id="L-737"><a href="#L-737"><span class="linenos">737</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">)</span>
|
||||
</span><span id="L-738"><a href="#L-738"><span class="linenos">738</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
|
||||
</span><span id="L-739"><a href="#L-739"><span class="linenos">739</span></a> <span class="ow">and</span> <span class="n">_is_subquery_scope</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-740"><a href="#L-740"><span class="linenos">740</span></a> <span class="p">)</span>
|
||||
</span><span id="L-741"><a href="#L-741"><span class="linenos">741</span></a> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">UDTF</span><span class="p">)</span>
|
||||
</span><span id="L-742"><a href="#L-742"><span class="linenos">742</span></a> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subqueryable</span><span class="p">)</span>
|
||||
</span><span id="L-743"><a href="#L-743"><span class="linenos">743</span></a> <span class="p">):</span>
|
||||
</span><span id="L-744"><a href="#L-744"><span class="linenos">744</span></a> <span class="n">prune</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="L-698"><a href="#L-698"><span class="linenos">698</span></a> <span class="c1"># append the final child_scope yielded</span>
|
||||
</span><span id="L-699"><a href="#L-699"><span class="linenos">699</span></a> <span class="n">scopes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">child_scope</span><span class="p">)</span>
|
||||
</span><span id="L-700"><a href="#L-700"><span class="linenos">700</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">table_scopes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">child_scope</span><span class="p">)</span>
|
||||
</span><span id="L-701"><a href="#L-701"><span class="linenos">701</span></a>
|
||||
</span><span id="L-702"><a href="#L-702"><span class="linenos">702</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">sources</span><span class="p">)</span>
|
||||
</span><span id="L-703"><a href="#L-703"><span class="linenos">703</span></a>
|
||||
</span><span id="L-704"><a href="#L-704"><span class="linenos">704</span></a>
|
||||
</span><span id="L-705"><a href="#L-705"><span class="linenos">705</span></a><span class="k">def</span> <span class="nf">_traverse_subqueries</span><span class="p">(</span><span class="n">scope</span><span class="p">):</span>
|
||||
</span><span id="L-706"><a href="#L-706"><span class="linenos">706</span></a> <span class="k">for</span> <span class="n">subquery</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">subqueries</span><span class="p">:</span>
|
||||
</span><span id="L-707"><a href="#L-707"><span class="linenos">707</span></a> <span class="n">top</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-708"><a href="#L-708"><span class="linenos">708</span></a> <span class="k">for</span> <span class="n">child_scope</span> <span class="ow">in</span> <span class="n">_traverse_scope</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">branch</span><span class="p">(</span><span class="n">subquery</span><span class="p">,</span> <span class="n">scope_type</span><span class="o">=</span><span class="n">ScopeType</span><span class="o">.</span><span class="n">SUBQUERY</span><span class="p">)):</span>
|
||||
</span><span id="L-709"><a href="#L-709"><span class="linenos">709</span></a> <span class="k">yield</span> <span class="n">child_scope</span>
|
||||
</span><span id="L-710"><a href="#L-710"><span class="linenos">710</span></a> <span class="n">top</span> <span class="o">=</span> <span class="n">child_scope</span>
|
||||
</span><span id="L-711"><a href="#L-711"><span class="linenos">711</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">subquery_scopes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">top</span><span class="p">)</span>
|
||||
</span><span id="L-712"><a href="#L-712"><span class="linenos">712</span></a>
|
||||
</span><span id="L-713"><a href="#L-713"><span class="linenos">713</span></a>
|
||||
</span><span id="L-714"><a href="#L-714"><span class="linenos">714</span></a><span class="k">def</span> <span class="nf">walk_in_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">bfs</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span>
|
||||
</span><span id="L-715"><a href="#L-715"><span class="linenos">715</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-716"><a href="#L-716"><span class="linenos">716</span></a><span class="sd"> Returns a generator object which visits all nodes in the syntrax tree, stopping at</span>
|
||||
</span><span id="L-717"><a href="#L-717"><span class="linenos">717</span></a><span class="sd"> nodes that start child scopes.</span>
|
||||
</span><span id="L-718"><a href="#L-718"><span class="linenos">718</span></a>
|
||||
</span><span id="L-719"><a href="#L-719"><span class="linenos">719</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-720"><a href="#L-720"><span class="linenos">720</span></a><span class="sd"> expression (exp.Expression):</span>
|
||||
</span><span id="L-721"><a href="#L-721"><span class="linenos">721</span></a><span class="sd"> bfs (bool): if set to True the BFS traversal order will be applied,</span>
|
||||
</span><span id="L-722"><a href="#L-722"><span class="linenos">722</span></a><span class="sd"> otherwise the DFS traversal will be used instead.</span>
|
||||
</span><span id="L-723"><a href="#L-723"><span class="linenos">723</span></a>
|
||||
</span><span id="L-724"><a href="#L-724"><span class="linenos">724</span></a><span class="sd"> Yields:</span>
|
||||
</span><span id="L-725"><a href="#L-725"><span class="linenos">725</span></a><span class="sd"> tuple[exp.Expression, Optional[exp.Expression], str]: node, parent, arg key</span>
|
||||
</span><span id="L-726"><a href="#L-726"><span class="linenos">726</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-727"><a href="#L-727"><span class="linenos">727</span></a> <span class="c1"># We'll use this variable to pass state into the dfs generator.</span>
|
||||
</span><span id="L-728"><a href="#L-728"><span class="linenos">728</span></a> <span class="c1"># Whenever we set it to True, we exclude a subtree from traversal.</span>
|
||||
</span><span id="L-729"><a href="#L-729"><span class="linenos">729</span></a> <span class="n">prune</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="L-730"><a href="#L-730"><span class="linenos">730</span></a>
|
||||
</span><span id="L-731"><a href="#L-731"><span class="linenos">731</span></a> <span class="k">for</span> <span class="n">node</span><span class="p">,</span> <span class="n">parent</span><span class="p">,</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">bfs</span><span class="o">=</span><span class="n">bfs</span><span class="p">,</span> <span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="o">*</span><span class="n">_</span><span class="p">:</span> <span class="n">prune</span><span class="p">):</span>
|
||||
</span><span id="L-732"><a href="#L-732"><span class="linenos">732</span></a> <span class="n">prune</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="L-733"><a href="#L-733"><span class="linenos">733</span></a>
|
||||
</span><span id="L-734"><a href="#L-734"><span class="linenos">734</span></a> <span class="k">yield</span> <span class="n">node</span><span class="p">,</span> <span class="n">parent</span><span class="p">,</span> <span class="n">key</span>
|
||||
</span><span id="L-735"><a href="#L-735"><span class="linenos">735</span></a>
|
||||
</span><span id="L-736"><a href="#L-736"><span class="linenos">736</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">is</span> <span class="n">expression</span><span class="p">:</span>
|
||||
</span><span id="L-737"><a href="#L-737"><span class="linenos">737</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-738"><a href="#L-738"><span class="linenos">738</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-739"><a href="#L-739"><span class="linenos">739</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">CTE</span><span class="p">)</span>
|
||||
</span><span id="L-740"><a href="#L-740"><span class="linenos">740</span></a> <span class="ow">or</span> <span class="p">(</span>
|
||||
</span><span id="L-741"><a href="#L-741"><span class="linenos">741</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">)</span>
|
||||
</span><span id="L-742"><a href="#L-742"><span class="linenos">742</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">))</span>
|
||||
</span><span id="L-743"><a href="#L-743"><span class="linenos">743</span></a> <span class="ow">and</span> <span class="n">_is_derived_table</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="L-744"><a href="#L-744"><span class="linenos">744</span></a> <span class="p">)</span>
|
||||
</span><span id="L-745"><a href="#L-745"><span class="linenos">745</span></a> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">UDTF</span><span class="p">)</span>
|
||||
</span><span id="L-746"><a href="#L-746"><span class="linenos">746</span></a> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subqueryable</span><span class="p">)</span>
|
||||
</span><span id="L-747"><a href="#L-747"><span class="linenos">747</span></a> <span class="p">):</span>
|
||||
</span><span id="L-748"><a href="#L-748"><span class="linenos">748</span></a> <span class="n">prune</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="L-749"><a href="#L-749"><span class="linenos">749</span></a>
|
||||
</span><span id="L-750"><a href="#L-750"><span class="linenos">750</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">UDTF</span><span class="p">)):</span>
|
||||
</span><span id="L-751"><a href="#L-751"><span class="linenos">751</span></a> <span class="c1"># The following args are not actually in the inner scope, so we should visit them</span>
|
||||
</span><span id="L-752"><a href="#L-752"><span class="linenos">752</span></a> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="s2">"laterals"</span><span class="p">,</span> <span class="s2">"pivots"</span><span class="p">):</span>
|
||||
</span><span id="L-753"><a href="#L-753"><span class="linenos">753</span></a> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]:</span>
|
||||
</span><span id="L-754"><a href="#L-754"><span class="linenos">754</span></a> <span class="k">yield from</span> <span class="n">walk_in_scope</span><span class="p">(</span><span class="n">arg</span><span class="p">,</span> <span class="n">bfs</span><span class="o">=</span><span class="n">bfs</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1213,8 +1223,8 @@
|
|||
</span><span id="Scope-125"><a href="#Scope-125"><span class="linenos">125</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_ctes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="Scope-126"><a href="#Scope-126"><span class="linenos">126</span></a> <span class="k">elif</span> <span class="p">(</span>
|
||||
</span><span id="Scope-127"><a href="#Scope-127"><span class="linenos">127</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">)</span>
|
||||
</span><span id="Scope-128"><a href="#Scope-128"><span class="linenos">128</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
|
||||
</span><span id="Scope-129"><a href="#Scope-129"><span class="linenos">129</span></a> <span class="ow">and</span> <span class="n">_is_subquery_scope</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="Scope-128"><a href="#Scope-128"><span class="linenos">128</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">))</span>
|
||||
</span><span id="Scope-129"><a href="#Scope-129"><span class="linenos">129</span></a> <span class="ow">and</span> <span class="n">_is_derived_table</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="Scope-130"><a href="#Scope-130"><span class="linenos">130</span></a> <span class="p">):</span>
|
||||
</span><span id="Scope-131"><a href="#Scope-131"><span class="linenos">131</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_derived_tables</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="Scope-132"><a href="#Scope-132"><span class="linenos">132</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subqueryable</span><span class="p">):</span>
|
||||
|
|
@ -2660,41 +2670,47 @@ incomplete properties which is confusing.</p>
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#walk_in_scope"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="walk_in_scope-711"><a href="#walk_in_scope-711"><span class="linenos">711</span></a><span class="k">def</span> <span class="nf">walk_in_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">bfs</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span>
|
||||
</span><span id="walk_in_scope-712"><a href="#walk_in_scope-712"><span class="linenos">712</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="walk_in_scope-713"><a href="#walk_in_scope-713"><span class="linenos">713</span></a><span class="sd"> Returns a generator object which visits all nodes in the syntrax tree, stopping at</span>
|
||||
</span><span id="walk_in_scope-714"><a href="#walk_in_scope-714"><span class="linenos">714</span></a><span class="sd"> nodes that start child scopes.</span>
|
||||
</span><span id="walk_in_scope-715"><a href="#walk_in_scope-715"><span class="linenos">715</span></a>
|
||||
</span><span id="walk_in_scope-716"><a href="#walk_in_scope-716"><span class="linenos">716</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="walk_in_scope-717"><a href="#walk_in_scope-717"><span class="linenos">717</span></a><span class="sd"> expression (exp.Expression):</span>
|
||||
</span><span id="walk_in_scope-718"><a href="#walk_in_scope-718"><span class="linenos">718</span></a><span class="sd"> bfs (bool): if set to True the BFS traversal order will be applied,</span>
|
||||
</span><span id="walk_in_scope-719"><a href="#walk_in_scope-719"><span class="linenos">719</span></a><span class="sd"> otherwise the DFS traversal will be used instead.</span>
|
||||
</span><span id="walk_in_scope-720"><a href="#walk_in_scope-720"><span class="linenos">720</span></a>
|
||||
</span><span id="walk_in_scope-721"><a href="#walk_in_scope-721"><span class="linenos">721</span></a><span class="sd"> Yields:</span>
|
||||
</span><span id="walk_in_scope-722"><a href="#walk_in_scope-722"><span class="linenos">722</span></a><span class="sd"> tuple[exp.Expression, Optional[exp.Expression], str]: node, parent, arg key</span>
|
||||
</span><span id="walk_in_scope-723"><a href="#walk_in_scope-723"><span class="linenos">723</span></a><span class="sd"> """</span>
|
||||
</span><span id="walk_in_scope-724"><a href="#walk_in_scope-724"><span class="linenos">724</span></a> <span class="c1"># We'll use this variable to pass state into the dfs generator.</span>
|
||||
</span><span id="walk_in_scope-725"><a href="#walk_in_scope-725"><span class="linenos">725</span></a> <span class="c1"># Whenever we set it to True, we exclude a subtree from traversal.</span>
|
||||
</span><span id="walk_in_scope-726"><a href="#walk_in_scope-726"><span class="linenos">726</span></a> <span class="n">prune</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="walk_in_scope-727"><a href="#walk_in_scope-727"><span class="linenos">727</span></a>
|
||||
</span><span id="walk_in_scope-728"><a href="#walk_in_scope-728"><span class="linenos">728</span></a> <span class="k">for</span> <span class="n">node</span><span class="p">,</span> <span class="n">parent</span><span class="p">,</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">bfs</span><span class="o">=</span><span class="n">bfs</span><span class="p">,</span> <span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="o">*</span><span class="n">_</span><span class="p">:</span> <span class="n">prune</span><span class="p">):</span>
|
||||
</span><span id="walk_in_scope-729"><a href="#walk_in_scope-729"><span class="linenos">729</span></a> <span class="n">prune</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="walk_in_scope-730"><a href="#walk_in_scope-730"><span class="linenos">730</span></a>
|
||||
</span><span id="walk_in_scope-731"><a href="#walk_in_scope-731"><span class="linenos">731</span></a> <span class="k">yield</span> <span class="n">node</span><span class="p">,</span> <span class="n">parent</span><span class="p">,</span> <span class="n">key</span>
|
||||
</span><span id="walk_in_scope-732"><a href="#walk_in_scope-732"><span class="linenos">732</span></a>
|
||||
</span><span id="walk_in_scope-733"><a href="#walk_in_scope-733"><span class="linenos">733</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">is</span> <span class="n">expression</span><span class="p">:</span>
|
||||
</span><span id="walk_in_scope-734"><a href="#walk_in_scope-734"><span class="linenos">734</span></a> <span class="k">continue</span>
|
||||
</span><span id="walk_in_scope-735"><a href="#walk_in_scope-735"><span class="linenos">735</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="walk_in_scope-736"><a href="#walk_in_scope-736"><span class="linenos">736</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">CTE</span><span class="p">)</span>
|
||||
</span><span id="walk_in_scope-737"><a href="#walk_in_scope-737"><span class="linenos">737</span></a> <span class="ow">or</span> <span class="p">(</span>
|
||||
</span><span id="walk_in_scope-738"><a href="#walk_in_scope-738"><span class="linenos">738</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">)</span>
|
||||
</span><span id="walk_in_scope-739"><a href="#walk_in_scope-739"><span class="linenos">739</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
|
||||
</span><span id="walk_in_scope-740"><a href="#walk_in_scope-740"><span class="linenos">740</span></a> <span class="ow">and</span> <span class="n">_is_subquery_scope</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="walk_in_scope-741"><a href="#walk_in_scope-741"><span class="linenos">741</span></a> <span class="p">)</span>
|
||||
</span><span id="walk_in_scope-742"><a href="#walk_in_scope-742"><span class="linenos">742</span></a> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">UDTF</span><span class="p">)</span>
|
||||
</span><span id="walk_in_scope-743"><a href="#walk_in_scope-743"><span class="linenos">743</span></a> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subqueryable</span><span class="p">)</span>
|
||||
</span><span id="walk_in_scope-744"><a href="#walk_in_scope-744"><span class="linenos">744</span></a> <span class="p">):</span>
|
||||
</span><span id="walk_in_scope-745"><a href="#walk_in_scope-745"><span class="linenos">745</span></a> <span class="n">prune</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="walk_in_scope-715"><a href="#walk_in_scope-715"><span class="linenos">715</span></a><span class="k">def</span> <span class="nf">walk_in_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">bfs</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span>
|
||||
</span><span id="walk_in_scope-716"><a href="#walk_in_scope-716"><span class="linenos">716</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="walk_in_scope-717"><a href="#walk_in_scope-717"><span class="linenos">717</span></a><span class="sd"> Returns a generator object which visits all nodes in the syntrax tree, stopping at</span>
|
||||
</span><span id="walk_in_scope-718"><a href="#walk_in_scope-718"><span class="linenos">718</span></a><span class="sd"> nodes that start child scopes.</span>
|
||||
</span><span id="walk_in_scope-719"><a href="#walk_in_scope-719"><span class="linenos">719</span></a>
|
||||
</span><span id="walk_in_scope-720"><a href="#walk_in_scope-720"><span class="linenos">720</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="walk_in_scope-721"><a href="#walk_in_scope-721"><span class="linenos">721</span></a><span class="sd"> expression (exp.Expression):</span>
|
||||
</span><span id="walk_in_scope-722"><a href="#walk_in_scope-722"><span class="linenos">722</span></a><span class="sd"> bfs (bool): if set to True the BFS traversal order will be applied,</span>
|
||||
</span><span id="walk_in_scope-723"><a href="#walk_in_scope-723"><span class="linenos">723</span></a><span class="sd"> otherwise the DFS traversal will be used instead.</span>
|
||||
</span><span id="walk_in_scope-724"><a href="#walk_in_scope-724"><span class="linenos">724</span></a>
|
||||
</span><span id="walk_in_scope-725"><a href="#walk_in_scope-725"><span class="linenos">725</span></a><span class="sd"> Yields:</span>
|
||||
</span><span id="walk_in_scope-726"><a href="#walk_in_scope-726"><span class="linenos">726</span></a><span class="sd"> tuple[exp.Expression, Optional[exp.Expression], str]: node, parent, arg key</span>
|
||||
</span><span id="walk_in_scope-727"><a href="#walk_in_scope-727"><span class="linenos">727</span></a><span class="sd"> """</span>
|
||||
</span><span id="walk_in_scope-728"><a href="#walk_in_scope-728"><span class="linenos">728</span></a> <span class="c1"># We'll use this variable to pass state into the dfs generator.</span>
|
||||
</span><span id="walk_in_scope-729"><a href="#walk_in_scope-729"><span class="linenos">729</span></a> <span class="c1"># Whenever we set it to True, we exclude a subtree from traversal.</span>
|
||||
</span><span id="walk_in_scope-730"><a href="#walk_in_scope-730"><span class="linenos">730</span></a> <span class="n">prune</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="walk_in_scope-731"><a href="#walk_in_scope-731"><span class="linenos">731</span></a>
|
||||
</span><span id="walk_in_scope-732"><a href="#walk_in_scope-732"><span class="linenos">732</span></a> <span class="k">for</span> <span class="n">node</span><span class="p">,</span> <span class="n">parent</span><span class="p">,</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">bfs</span><span class="o">=</span><span class="n">bfs</span><span class="p">,</span> <span class="n">prune</span><span class="o">=</span><span class="k">lambda</span> <span class="o">*</span><span class="n">_</span><span class="p">:</span> <span class="n">prune</span><span class="p">):</span>
|
||||
</span><span id="walk_in_scope-733"><a href="#walk_in_scope-733"><span class="linenos">733</span></a> <span class="n">prune</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="walk_in_scope-734"><a href="#walk_in_scope-734"><span class="linenos">734</span></a>
|
||||
</span><span id="walk_in_scope-735"><a href="#walk_in_scope-735"><span class="linenos">735</span></a> <span class="k">yield</span> <span class="n">node</span><span class="p">,</span> <span class="n">parent</span><span class="p">,</span> <span class="n">key</span>
|
||||
</span><span id="walk_in_scope-736"><a href="#walk_in_scope-736"><span class="linenos">736</span></a>
|
||||
</span><span id="walk_in_scope-737"><a href="#walk_in_scope-737"><span class="linenos">737</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">is</span> <span class="n">expression</span><span class="p">:</span>
|
||||
</span><span id="walk_in_scope-738"><a href="#walk_in_scope-738"><span class="linenos">738</span></a> <span class="k">continue</span>
|
||||
</span><span id="walk_in_scope-739"><a href="#walk_in_scope-739"><span class="linenos">739</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="walk_in_scope-740"><a href="#walk_in_scope-740"><span class="linenos">740</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">CTE</span><span class="p">)</span>
|
||||
</span><span id="walk_in_scope-741"><a href="#walk_in_scope-741"><span class="linenos">741</span></a> <span class="ow">or</span> <span class="p">(</span>
|
||||
</span><span id="walk_in_scope-742"><a href="#walk_in_scope-742"><span class="linenos">742</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">)</span>
|
||||
</span><span id="walk_in_scope-743"><a href="#walk_in_scope-743"><span class="linenos">743</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">))</span>
|
||||
</span><span id="walk_in_scope-744"><a href="#walk_in_scope-744"><span class="linenos">744</span></a> <span class="ow">and</span> <span class="n">_is_derived_table</span><span class="p">(</span><span class="n">node</span><span class="p">)</span>
|
||||
</span><span id="walk_in_scope-745"><a href="#walk_in_scope-745"><span class="linenos">745</span></a> <span class="p">)</span>
|
||||
</span><span id="walk_in_scope-746"><a href="#walk_in_scope-746"><span class="linenos">746</span></a> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">UDTF</span><span class="p">)</span>
|
||||
</span><span id="walk_in_scope-747"><a href="#walk_in_scope-747"><span class="linenos">747</span></a> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subqueryable</span><span class="p">)</span>
|
||||
</span><span id="walk_in_scope-748"><a href="#walk_in_scope-748"><span class="linenos">748</span></a> <span class="p">):</span>
|
||||
</span><span id="walk_in_scope-749"><a href="#walk_in_scope-749"><span class="linenos">749</span></a> <span class="n">prune</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="walk_in_scope-750"><a href="#walk_in_scope-750"><span class="linenos">750</span></a>
|
||||
</span><span id="walk_in_scope-751"><a href="#walk_in_scope-751"><span class="linenos">751</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">UDTF</span><span class="p">)):</span>
|
||||
</span><span id="walk_in_scope-752"><a href="#walk_in_scope-752"><span class="linenos">752</span></a> <span class="c1"># The following args are not actually in the inner scope, so we should visit them</span>
|
||||
</span><span id="walk_in_scope-753"><a href="#walk_in_scope-753"><span class="linenos">753</span></a> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="s2">"laterals"</span><span class="p">,</span> <span class="s2">"pivots"</span><span class="p">):</span>
|
||||
</span><span id="walk_in_scope-754"><a href="#walk_in_scope-754"><span class="linenos">754</span></a> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]:</span>
|
||||
</span><span id="walk_in_scope-755"><a href="#walk_in_scope-755"><span class="linenos">755</span></a> <span class="k">yield from</span> <span class="n">walk_in_scope</span><span class="p">(</span><span class="n">arg</span><span class="p">,</span> <span class="n">bfs</span><span class="o">=</span><span class="n">bfs</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
17672
docs/sqlglot/parser.html
17672
docs/sqlglot/parser.html
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load diff
|
|
@ -114,7 +114,7 @@ class Column:
|
|||
return self.inverse_binary_op(exp.Or, other)
|
||||
|
||||
@classmethod
|
||||
def ensure_col(cls, value: t.Optional[t.Union[ColumnOrLiteral, exp.Expression]]):
|
||||
def ensure_col(cls, value: t.Optional[t.Union[ColumnOrLiteral, exp.Expression]]) -> Column:
|
||||
return cls(value)
|
||||
|
||||
@classmethod
|
||||
|
|
@ -259,7 +259,7 @@ class Column:
|
|||
new_expression = exp.Not(this=exp.Is(this=self.column_expression, expression=exp.Null()))
|
||||
return Column(new_expression)
|
||||
|
||||
def cast(self, dataType: t.Union[str, DataType]):
|
||||
def cast(self, dataType: t.Union[str, DataType]) -> Column:
|
||||
"""
|
||||
Functionality Difference: PySpark cast accepts a datatype instance of the datatype class
|
||||
Sqlglot doesn't currently replicate this class so it only accepts a string
|
||||
|
|
|
|||
|
|
@ -600,8 +600,13 @@ def months_between(
|
|||
date1: ColumnOrName, date2: ColumnOrName, roundOff: t.Optional[bool] = None
|
||||
) -> Column:
|
||||
if roundOff is None:
|
||||
return Column.invoke_anonymous_function(date1, "MONTHS_BETWEEN", date2)
|
||||
return Column.invoke_anonymous_function(date1, "MONTHS_BETWEEN", date2, roundOff)
|
||||
return Column.invoke_expression_over_column(
|
||||
date1, expression.MonthsBetween, expression=date2
|
||||
)
|
||||
|
||||
return Column.invoke_expression_over_column(
|
||||
date1, expression.MonthsBetween, expression=date2, roundoff=roundOff
|
||||
)
|
||||
|
||||
|
||||
def to_date(col: ColumnOrName, format: t.Optional[str] = None) -> Column:
|
||||
|
|
@ -614,8 +619,9 @@ def to_date(col: ColumnOrName, format: t.Optional[str] = None) -> Column:
|
|||
|
||||
def to_timestamp(col: ColumnOrName, format: t.Optional[str] = None) -> Column:
|
||||
if format is not None:
|
||||
return Column.invoke_anonymous_function(col, "TO_TIMESTAMP", lit(format))
|
||||
return Column.invoke_anonymous_function(col, "TO_TIMESTAMP")
|
||||
return Column.invoke_expression_over_column(col, expression.StrToTime, format=lit(format))
|
||||
|
||||
return Column.ensure_col(col).cast("timestamp")
|
||||
|
||||
|
||||
def trunc(col: ColumnOrName, format: str) -> Column:
|
||||
|
|
@ -875,8 +881,16 @@ def regexp_extract(str: ColumnOrName, pattern: str, idx: t.Optional[int] = None)
|
|||
)
|
||||
|
||||
|
||||
def regexp_replace(str: ColumnOrName, pattern: str, replacement: str) -> Column:
|
||||
return Column.invoke_anonymous_function(str, "REGEXP_REPLACE", lit(pattern), lit(replacement))
|
||||
def regexp_replace(
|
||||
str: ColumnOrName, pattern: str, replacement: str, position: t.Optional[int] = None
|
||||
) -> Column:
|
||||
return Column.invoke_expression_over_column(
|
||||
str,
|
||||
expression.RegexpReplace,
|
||||
expression=lit(pattern),
|
||||
replacement=lit(replacement),
|
||||
position=position,
|
||||
)
|
||||
|
||||
|
||||
def initcap(col: ColumnOrName) -> Column:
|
||||
|
|
@ -1186,7 +1200,9 @@ def transform(
|
|||
f: t.Union[t.Callable[[Column], Column], t.Callable[[Column, Column], Column]],
|
||||
) -> Column:
|
||||
f_expression = _get_lambda_from_func(f)
|
||||
return Column.invoke_anonymous_function(col, "TRANSFORM", Column(f_expression))
|
||||
return Column.invoke_expression_over_column(
|
||||
col, expression.Transform, expression=Column(f_expression)
|
||||
)
|
||||
|
||||
|
||||
def exists(col: ColumnOrName, f: t.Callable[[Column], Column]) -> Column:
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ from sqlglot import exp, generator, parser, tokens, transforms
|
|||
from sqlglot._typing import E
|
||||
from sqlglot.dialects.dialect import (
|
||||
Dialect,
|
||||
binary_from_function,
|
||||
datestrtodate_sql,
|
||||
format_time_lambda,
|
||||
inline_array_sql,
|
||||
|
|
@ -15,6 +16,7 @@ from sqlglot.dialects.dialect import (
|
|||
min_or_least,
|
||||
no_ilike_sql,
|
||||
parse_date_delta_with_interval,
|
||||
regexp_replace_sql,
|
||||
rename_func,
|
||||
timestrtotime_sql,
|
||||
ts_or_ds_to_date_sql,
|
||||
|
|
@ -39,7 +41,7 @@ def _date_add_sql(
|
|||
|
||||
|
||||
def _derived_table_values_to_unnest(self: generator.Generator, expression: exp.Values) -> str:
|
||||
if not isinstance(expression.unnest().parent, exp.From):
|
||||
if not expression.find_ancestor(exp.From, exp.Join):
|
||||
return self.values_sql(expression)
|
||||
|
||||
alias = expression.args.get("alias")
|
||||
|
|
@ -279,7 +281,7 @@ class BigQuery(Dialect):
|
|||
),
|
||||
"DATETIME_ADD": parse_date_delta_with_interval(exp.DatetimeAdd),
|
||||
"DATETIME_SUB": parse_date_delta_with_interval(exp.DatetimeSub),
|
||||
"DIV": lambda args: exp.IntDiv(this=seq_get(args, 0), expression=seq_get(args, 1)),
|
||||
"DIV": binary_from_function(exp.IntDiv),
|
||||
"GENERATE_ARRAY": exp.GenerateSeries.from_arg_list,
|
||||
"MD5": exp.MD5Digest.from_arg_list,
|
||||
"TO_HEX": _parse_to_hex,
|
||||
|
|
@ -415,6 +417,7 @@ class BigQuery(Dialect):
|
|||
e.args.get("position"),
|
||||
e.args.get("occurrence"),
|
||||
),
|
||||
exp.RegexpReplace: regexp_replace_sql,
|
||||
exp.RegexpLike: rename_func("REGEXP_CONTAINS"),
|
||||
exp.ReturnsProperty: _returnsproperty_sql,
|
||||
exp.Select: transforms.preprocess(
|
||||
|
|
|
|||
|
|
@ -64,6 +64,7 @@ class ClickHouse(Dialect):
|
|||
"MAP": parse_var_map,
|
||||
"MATCH": exp.RegexpLike.from_arg_list,
|
||||
"UNIQ": exp.ApproxDistinct.from_arg_list,
|
||||
"XOR": lambda args: exp.Xor(expressions=args),
|
||||
}
|
||||
|
||||
FUNCTIONS_WITH_ALIASED_ARGS = {*parser.Parser.FUNCTIONS_WITH_ALIASED_ARGS, "TUPLE"}
|
||||
|
|
@ -95,6 +96,7 @@ class ClickHouse(Dialect):
|
|||
TokenType.ASOF,
|
||||
TokenType.ANTI,
|
||||
TokenType.SEMI,
|
||||
TokenType.ARRAY,
|
||||
}
|
||||
|
||||
TABLE_ALIAS_TOKENS = {*parser.Parser.TABLE_ALIAS_TOKENS} - {
|
||||
|
|
@ -103,6 +105,7 @@ class ClickHouse(Dialect):
|
|||
TokenType.ANTI,
|
||||
TokenType.SETTINGS,
|
||||
TokenType.FORMAT,
|
||||
TokenType.ARRAY,
|
||||
}
|
||||
|
||||
LOG_DEFAULTS_TO_LN = True
|
||||
|
|
@ -160,8 +163,11 @@ class ClickHouse(Dialect):
|
|||
schema: bool = False,
|
||||
joins: bool = False,
|
||||
alias_tokens: t.Optional[t.Collection[TokenType]] = None,
|
||||
parse_bracket: bool = False,
|
||||
) -> t.Optional[exp.Expression]:
|
||||
this = super()._parse_table(schema=schema, joins=joins, alias_tokens=alias_tokens)
|
||||
this = super()._parse_table(
|
||||
schema=schema, joins=joins, alias_tokens=alias_tokens, parse_bracket=parse_bracket
|
||||
)
|
||||
|
||||
if self._match(TokenType.FINAL):
|
||||
this = self.expression(exp.Final, this=this)
|
||||
|
|
@ -204,8 +210,10 @@ class ClickHouse(Dialect):
|
|||
self._match_set(self.JOIN_KINDS) and self._prev,
|
||||
)
|
||||
|
||||
def _parse_join(self, skip_join_token: bool = False) -> t.Optional[exp.Join]:
|
||||
join = super()._parse_join(skip_join_token)
|
||||
def _parse_join(
|
||||
self, skip_join_token: bool = False, parse_bracket: bool = False
|
||||
) -> t.Optional[exp.Join]:
|
||||
join = super()._parse_join(skip_join_token=skip_join_token, parse_bracket=True)
|
||||
|
||||
if join:
|
||||
join.set("global", join.args.pop("method", None))
|
||||
|
|
@ -318,6 +326,7 @@ class ClickHouse(Dialect):
|
|||
exp.RegexpLike: lambda self, e: f"match({self.format_args(e.this, e.expression)})",
|
||||
exp.StrPosition: lambda self, e: f"position({self.format_args(e.this, e.args.get('substr'), e.args.get('position'))})",
|
||||
exp.VarMap: lambda self, e: _lower_func(var_map_sql(self, e)),
|
||||
exp.Xor: lambda self, e: self.func("xor", e.this, e.expression, *e.expressions),
|
||||
}
|
||||
|
||||
PROPERTIES_LOCATION = {
|
||||
|
|
|
|||
|
|
@ -12,6 +12,8 @@ from sqlglot.time import format_time
|
|||
from sqlglot.tokens import Token, Tokenizer, TokenType
|
||||
from sqlglot.trie import new_trie
|
||||
|
||||
B = t.TypeVar("B", bound=exp.Binary)
|
||||
|
||||
|
||||
class Dialects(str, Enum):
|
||||
DIALECT = ""
|
||||
|
|
@ -630,6 +632,16 @@ def regexp_extract_sql(self: Generator, expression: exp.RegexpExtract) -> str:
|
|||
)
|
||||
|
||||
|
||||
def regexp_replace_sql(self: Generator, expression: exp.RegexpReplace) -> str:
|
||||
bad_args = list(filter(expression.args.get, ("position", "occurrence", "parameters")))
|
||||
if bad_args:
|
||||
self.unsupported(f"REGEXP_REPLACE does not support the following arg(s): {bad_args}")
|
||||
|
||||
return self.func(
|
||||
"REGEXP_REPLACE", expression.this, expression.expression, expression.args["replacement"]
|
||||
)
|
||||
|
||||
|
||||
def pivot_column_names(aggregations: t.List[exp.Expression], dialect: DialectType) -> t.List[str]:
|
||||
names = []
|
||||
for agg in aggregations:
|
||||
|
|
@ -650,3 +662,7 @@ def pivot_column_names(aggregations: t.List[exp.Expression], dialect: DialectTyp
|
|||
names.append(agg_all_unquoted.sql(dialect=dialect, normalize_functions="lower"))
|
||||
|
||||
return names
|
||||
|
||||
|
||||
def binary_from_function(expr_type: t.Type[B]) -> t.Callable[[t.List], B]:
|
||||
return lambda args: expr_type(this=seq_get(args, 0), expression=seq_get(args, 1))
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ from sqlglot.dialects.dialect import (
|
|||
approx_count_distinct_sql,
|
||||
arrow_json_extract_scalar_sql,
|
||||
arrow_json_extract_sql,
|
||||
binary_from_function,
|
||||
date_trunc_to_time,
|
||||
datestrtodate_sql,
|
||||
format_time_lambda,
|
||||
|
|
@ -16,6 +17,7 @@ from sqlglot.dialects.dialect import (
|
|||
no_safe_divide_sql,
|
||||
pivot_column_names,
|
||||
regexp_extract_sql,
|
||||
regexp_replace_sql,
|
||||
rename_func,
|
||||
str_position_sql,
|
||||
str_to_time_sql,
|
||||
|
|
@ -103,7 +105,6 @@ class DuckDB(Dialect):
|
|||
class Tokenizer(tokens.Tokenizer):
|
||||
KEYWORDS = {
|
||||
**tokens.Tokenizer.KEYWORDS,
|
||||
"~": TokenType.RLIKE,
|
||||
":=": TokenType.EQ,
|
||||
"//": TokenType.DIV,
|
||||
"ATTACH": TokenType.COMMAND,
|
||||
|
|
@ -128,6 +129,11 @@ class DuckDB(Dialect):
|
|||
class Parser(parser.Parser):
|
||||
CONCAT_NULL_OUTPUTS_STRING = True
|
||||
|
||||
BITWISE = {
|
||||
**parser.Parser.BITWISE,
|
||||
TokenType.TILDA: exp.RegexpLike,
|
||||
}
|
||||
|
||||
FUNCTIONS = {
|
||||
**parser.Parser.FUNCTIONS,
|
||||
"ARRAY_LENGTH": exp.ArraySize.from_arg_list,
|
||||
|
|
@ -158,6 +164,7 @@ class DuckDB(Dialect):
|
|||
"STR_SPLIT_REGEX": exp.RegexpSplit.from_arg_list,
|
||||
"TO_TIMESTAMP": exp.UnixToTime.from_arg_list,
|
||||
"UNNEST": exp.Explode.from_arg_list,
|
||||
"XOR": binary_from_function(exp.BitwiseXor),
|
||||
}
|
||||
|
||||
TYPE_TOKENS = {
|
||||
|
|
@ -190,6 +197,7 @@ class DuckDB(Dialect):
|
|||
exp.ArraySize: rename_func("ARRAY_LENGTH"),
|
||||
exp.ArraySort: _array_sort_sql,
|
||||
exp.ArraySum: rename_func("LIST_SUM"),
|
||||
exp.BitwiseXor: lambda self, e: self.func("XOR", e.this, e.expression),
|
||||
exp.CommentColumnConstraint: no_comment_column_constraint_sql,
|
||||
exp.CurrentDate: lambda self, e: "CURRENT_DATE",
|
||||
exp.CurrentTime: lambda self, e: "CURRENT_TIME",
|
||||
|
|
@ -203,7 +211,7 @@ class DuckDB(Dialect):
|
|||
exp.DateFromParts: rename_func("MAKE_DATE"),
|
||||
exp.DateSub: _date_delta_sql,
|
||||
exp.DateDiff: lambda self, e: self.func(
|
||||
"DATE_DIFF", f"'{e.args.get('unit', 'day')}'", e.expression, e.this
|
||||
"DATE_DIFF", f"'{e.args.get('unit') or 'day'}'", e.expression, e.this
|
||||
),
|
||||
exp.DateStrToDate: datestrtodate_sql,
|
||||
exp.DateToDi: lambda self, e: f"CAST(STRFTIME({self.sql(e, 'this')}, {DuckDB.DATEINT_FORMAT}) AS INT)",
|
||||
|
|
@ -217,8 +225,15 @@ class DuckDB(Dialect):
|
|||
exp.JSONBExtractScalar: arrow_json_extract_scalar_sql,
|
||||
exp.LogicalOr: rename_func("BOOL_OR"),
|
||||
exp.LogicalAnd: rename_func("BOOL_AND"),
|
||||
exp.MonthsBetween: lambda self, e: self.func(
|
||||
"DATEDIFF",
|
||||
"'month'",
|
||||
exp.cast(e.expression, "timestamp"),
|
||||
exp.cast(e.this, "timestamp"),
|
||||
),
|
||||
exp.Properties: no_properties_sql,
|
||||
exp.RegexpExtract: regexp_extract_sql,
|
||||
exp.RegexpReplace: regexp_replace_sql,
|
||||
exp.RegexpLike: rename_func("REGEXP_MATCHES"),
|
||||
exp.RegexpSplit: rename_func("STR_SPLIT_REGEX"),
|
||||
exp.SafeDivide: no_safe_divide_sql,
|
||||
|
|
|
|||
|
|
@ -18,6 +18,7 @@ from sqlglot.dialects.dialect import (
|
|||
no_safe_divide_sql,
|
||||
no_trycast_sql,
|
||||
regexp_extract_sql,
|
||||
regexp_replace_sql,
|
||||
rename_func,
|
||||
right_to_substring_sql,
|
||||
strposition_to_locate_sql,
|
||||
|
|
@ -211,6 +212,7 @@ class Hive(Dialect):
|
|||
"ADD JAR": TokenType.COMMAND,
|
||||
"ADD JARS": TokenType.COMMAND,
|
||||
"MSCK REPAIR": TokenType.COMMAND,
|
||||
"REFRESH": TokenType.COMMAND,
|
||||
"WITH SERDEPROPERTIES": TokenType.SERDE_PROPERTIES,
|
||||
}
|
||||
|
||||
|
|
@ -270,6 +272,11 @@ class Hive(Dialect):
|
|||
"YEAR": lambda args: exp.Year(this=exp.TsOrDsToDate.from_arg_list(args)),
|
||||
}
|
||||
|
||||
FUNCTION_PARSERS = {
|
||||
**parser.Parser.FUNCTION_PARSERS,
|
||||
"TRANSFORM": lambda self: self._parse_transform(),
|
||||
}
|
||||
|
||||
PROPERTY_PARSERS = {
|
||||
**parser.Parser.PROPERTY_PARSERS,
|
||||
"WITH SERDEPROPERTIES": lambda self: exp.SerdeProperties(
|
||||
|
|
@ -277,6 +284,40 @@ class Hive(Dialect):
|
|||
),
|
||||
}
|
||||
|
||||
def _parse_transform(self) -> exp.Transform | exp.QueryTransform:
|
||||
args = self._parse_csv(self._parse_lambda)
|
||||
self._match_r_paren()
|
||||
|
||||
row_format_before = self._parse_row_format(match_row=True)
|
||||
|
||||
record_writer = None
|
||||
if self._match_text_seq("RECORDWRITER"):
|
||||
record_writer = self._parse_string()
|
||||
|
||||
if not self._match(TokenType.USING):
|
||||
return exp.Transform.from_arg_list(args)
|
||||
|
||||
command_script = self._parse_string()
|
||||
|
||||
self._match(TokenType.ALIAS)
|
||||
schema = self._parse_schema()
|
||||
|
||||
row_format_after = self._parse_row_format(match_row=True)
|
||||
record_reader = None
|
||||
if self._match_text_seq("RECORDREADER"):
|
||||
record_reader = self._parse_string()
|
||||
|
||||
return self.expression(
|
||||
exp.QueryTransform,
|
||||
expressions=args,
|
||||
command_script=command_script,
|
||||
schema=schema,
|
||||
row_format_before=row_format_before,
|
||||
record_writer=record_writer,
|
||||
row_format_after=row_format_after,
|
||||
record_reader=record_reader,
|
||||
)
|
||||
|
||||
def _parse_types(
|
||||
self, check_func: bool = False, schema: bool = False
|
||||
) -> t.Optional[exp.Expression]:
|
||||
|
|
@ -363,11 +404,13 @@ class Hive(Dialect):
|
|||
exp.Max: max_or_greatest,
|
||||
exp.MD5Digest: lambda self, e: self.func("UNHEX", self.func("MD5", e.this)),
|
||||
exp.Min: min_or_least,
|
||||
exp.MonthsBetween: lambda self, e: self.func("MONTHS_BETWEEN", e.this, e.expression),
|
||||
exp.VarMap: var_map_sql,
|
||||
exp.Create: create_with_partitions_sql,
|
||||
exp.Quantile: rename_func("PERCENTILE"),
|
||||
exp.ApproxQuantile: rename_func("PERCENTILE_APPROX"),
|
||||
exp.RegexpExtract: regexp_extract_sql,
|
||||
exp.RegexpReplace: regexp_replace_sql,
|
||||
exp.RegexpLike: lambda self, e: self.binary(e, "RLIKE"),
|
||||
exp.RegexpSplit: rename_func("SPLIT"),
|
||||
exp.Right: right_to_substring_sql,
|
||||
|
|
@ -396,7 +439,6 @@ class Hive(Dialect):
|
|||
exp.UnixToTime: rename_func("FROM_UNIXTIME"),
|
||||
exp.UnixToTimeStr: rename_func("FROM_UNIXTIME"),
|
||||
exp.PartitionedByProperty: lambda self, e: f"PARTITIONED BY {self.sql(e, 'this')}",
|
||||
exp.RowFormatSerdeProperty: lambda self, e: f"ROW FORMAT SERDE {self.sql(e, 'this')}",
|
||||
exp.SerdeProperties: lambda self, e: self.properties(e, prefix="WITH SERDEPROPERTIES"),
|
||||
exp.NumberToStr: rename_func("FORMAT_NUMBER"),
|
||||
exp.LastDateOfMonth: rename_func("LAST_DAY"),
|
||||
|
|
@ -410,6 +452,11 @@ class Hive(Dialect):
|
|||
exp.VolatileProperty: exp.Properties.Location.UNSUPPORTED,
|
||||
}
|
||||
|
||||
def rowformatserdeproperty_sql(self, expression: exp.RowFormatSerdeProperty) -> str:
|
||||
serde_props = self.sql(expression, "serde_properties")
|
||||
serde_props = f" {serde_props}" if serde_props else ""
|
||||
return f"ROW FORMAT SERDE {self.sql(expression, 'this')}{serde_props}"
|
||||
|
||||
def arrayagg_sql(self, expression: exp.ArrayAgg) -> str:
|
||||
return self.func(
|
||||
"COLLECT_LIST",
|
||||
|
|
|
|||
|
|
@ -427,6 +427,7 @@ class MySQL(Dialect):
|
|||
TABLE_HINTS = True
|
||||
DUPLICATE_KEY_UPDATE_WITH_SET = False
|
||||
QUERY_HINT_SEP = " "
|
||||
VALUES_AS_TABLE = False
|
||||
|
||||
TRANSFORMS = {
|
||||
**generator.Generator.TRANSFORMS,
|
||||
|
|
@ -473,19 +474,32 @@ class MySQL(Dialect):
|
|||
|
||||
LIMIT_FETCH = "LIMIT"
|
||||
|
||||
# MySQL doesn't support many datatypes in cast.
|
||||
# https://dev.mysql.com/doc/refman/8.0/en/cast-functions.html#function_cast
|
||||
CAST_MAPPING = {
|
||||
exp.DataType.Type.BIGINT: "SIGNED",
|
||||
exp.DataType.Type.BOOLEAN: "SIGNED",
|
||||
exp.DataType.Type.INT: "SIGNED",
|
||||
exp.DataType.Type.TEXT: "CHAR",
|
||||
exp.DataType.Type.UBIGINT: "UNSIGNED",
|
||||
exp.DataType.Type.VARCHAR: "CHAR",
|
||||
}
|
||||
|
||||
def xor_sql(self, expression: exp.Xor) -> str:
|
||||
if expression.expressions:
|
||||
return self.expressions(expression, sep=" XOR ")
|
||||
return super().xor_sql(expression)
|
||||
|
||||
def jsonarraycontains_sql(self, expression: exp.JSONArrayContains) -> str:
|
||||
return f"{self.sql(expression, 'this')} MEMBER OF({self.sql(expression, 'expression')})"
|
||||
|
||||
def cast_sql(self, expression: exp.Cast, safe_prefix: t.Optional[str] = None) -> str:
|
||||
"""(U)BIGINT is not allowed in a CAST expression, so we use (UN)SIGNED instead."""
|
||||
if expression.to.this == exp.DataType.Type.BIGINT:
|
||||
to = "SIGNED"
|
||||
elif expression.to.this == exp.DataType.Type.UBIGINT:
|
||||
to = "UNSIGNED"
|
||||
else:
|
||||
return super().cast_sql(expression)
|
||||
to = self.CAST_MAPPING.get(expression.to.this)
|
||||
|
||||
return f"CAST({self.sql(expression, 'this')} AS {to})"
|
||||
if to:
|
||||
expression = expression.copy()
|
||||
expression.to.set("this", to)
|
||||
return super().cast_sql(expression)
|
||||
|
||||
def show_sql(self, expression: exp.Show) -> str:
|
||||
this = f" {expression.name}"
|
||||
|
|
|
|||
|
|
@ -282,7 +282,6 @@ class Postgres(Dialect):
|
|||
VAR_SINGLE_TOKENS = {"$"}
|
||||
|
||||
class Parser(parser.Parser):
|
||||
STRICT_CAST = False
|
||||
CONCAT_NULL_OUTPUTS_STRING = True
|
||||
|
||||
FUNCTIONS = {
|
||||
|
|
@ -318,6 +317,11 @@ class Postgres(Dialect):
|
|||
TokenType.LT_AT: binary_range_parser(exp.ArrayContained),
|
||||
}
|
||||
|
||||
STATEMENT_PARSERS = {
|
||||
**parser.Parser.STATEMENT_PARSERS,
|
||||
TokenType.END: lambda self: self._parse_commit_or_rollback(),
|
||||
}
|
||||
|
||||
def _parse_factor(self) -> t.Optional[exp.Expression]:
|
||||
return self._parse_tokens(self._parse_exponent, self.FACTOR)
|
||||
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@ import typing as t
|
|||
from sqlglot import exp, generator, parser, tokens, transforms
|
||||
from sqlglot.dialects.dialect import (
|
||||
Dialect,
|
||||
binary_from_function,
|
||||
date_trunc_to_time,
|
||||
format_time_lambda,
|
||||
if_sql,
|
||||
|
|
@ -198,6 +199,10 @@ class Presto(Dialect):
|
|||
**parser.Parser.FUNCTIONS,
|
||||
"APPROX_DISTINCT": exp.ApproxDistinct.from_arg_list,
|
||||
"APPROX_PERCENTILE": _approx_percentile,
|
||||
"BITWISE_AND": binary_from_function(exp.BitwiseAnd),
|
||||
"BITWISE_NOT": lambda args: exp.BitwiseNot(this=seq_get(args, 0)),
|
||||
"BITWISE_OR": binary_from_function(exp.BitwiseOr),
|
||||
"BITWISE_XOR": binary_from_function(exp.BitwiseXor),
|
||||
"CARDINALITY": exp.ArraySize.from_arg_list,
|
||||
"CONTAINS": exp.ArrayContains.from_arg_list,
|
||||
"DATE_ADD": lambda args: exp.DateAdd(
|
||||
|
|
|
|||
|
|
@ -27,6 +27,11 @@ class Redshift(Postgres):
|
|||
class Parser(Postgres.Parser):
|
||||
FUNCTIONS = {
|
||||
**Postgres.Parser.FUNCTIONS,
|
||||
"ADD_MONTHS": lambda args: exp.DateAdd(
|
||||
this=exp.TsOrDsToDate(this=seq_get(args, 0)),
|
||||
expression=seq_get(args, 1),
|
||||
unit=exp.var("month"),
|
||||
),
|
||||
"DATEADD": lambda args: exp.DateAdd(
|
||||
this=exp.TsOrDsToDate(this=seq_get(args, 2)),
|
||||
expression=seq_get(args, 1),
|
||||
|
|
@ -37,7 +42,6 @@ class Redshift(Postgres):
|
|||
expression=exp.TsOrDsToDate(this=seq_get(args, 1)),
|
||||
unit=seq_get(args, 0),
|
||||
),
|
||||
"NVL": exp.Coalesce.from_arg_list,
|
||||
"STRTOL": exp.FromBase.from_arg_list,
|
||||
}
|
||||
|
||||
|
|
@ -87,6 +91,7 @@ class Redshift(Postgres):
|
|||
LOCKING_READS_SUPPORTED = False
|
||||
RENAME_TABLE_WITH_DB = False
|
||||
QUERY_HINTS = False
|
||||
VALUES_AS_TABLE = False
|
||||
|
||||
TYPE_MAPPING = {
|
||||
**Postgres.Generator.TYPE_MAPPING,
|
||||
|
|
@ -129,40 +134,6 @@ class Redshift(Postgres):
|
|||
|
||||
RESERVED_KEYWORDS = {*Postgres.Generator.RESERVED_KEYWORDS, "snapshot", "type"}
|
||||
|
||||
def values_sql(self, expression: exp.Values) -> str:
|
||||
"""
|
||||
Converts `VALUES...` expression into a series of unions.
|
||||
|
||||
Note: If you have a lot of unions then this will result in a large number of recursive statements to
|
||||
evaluate the expression. You may need to increase `sys.setrecursionlimit` to run and it can also be
|
||||
very slow.
|
||||
"""
|
||||
|
||||
# The VALUES clause is still valid in an `INSERT INTO ..` statement, for example
|
||||
if not expression.find_ancestor(exp.From, exp.Join):
|
||||
return super().values_sql(expression)
|
||||
|
||||
column_names = expression.alias and expression.args["alias"].columns
|
||||
|
||||
selects = []
|
||||
rows = [tuple_exp.expressions for tuple_exp in expression.expressions]
|
||||
|
||||
for i, row in enumerate(rows):
|
||||
if i == 0 and column_names:
|
||||
row = [
|
||||
exp.alias_(value, column_name)
|
||||
for value, column_name in zip(row, column_names)
|
||||
]
|
||||
|
||||
selects.append(exp.Select(expressions=row))
|
||||
|
||||
subquery_expression: exp.Select | exp.Union = selects[0]
|
||||
if len(selects) > 1:
|
||||
for select in selects[1:]:
|
||||
subquery_expression = exp.union(subquery_expression, select, distinct=False)
|
||||
|
||||
return self.subquery_sql(subquery_expression.subquery(expression.alias))
|
||||
|
||||
def with_properties(self, properties: exp.Properties) -> str:
|
||||
"""Redshift doesn't have `WITH` as part of their with_properties so we remove it"""
|
||||
return self.properties(properties, prefix=" ", suffix="")
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ def _check_int(s: str) -> bool:
|
|||
|
||||
|
||||
# from https://docs.snowflake.com/en/sql-reference/functions/to_timestamp.html
|
||||
def _snowflake_to_timestamp(args: t.List) -> t.Union[exp.StrToTime, exp.UnixToTime]:
|
||||
def _parse_to_timestamp(args: t.List) -> t.Union[exp.StrToTime, exp.UnixToTime]:
|
||||
if len(args) == 2:
|
||||
first_arg, second_arg = args
|
||||
if second_arg.is_string:
|
||||
|
|
@ -137,7 +137,7 @@ def _parse_date_part(self: parser.Parser) -> t.Optional[exp.Expression]:
|
|||
|
||||
|
||||
# https://docs.snowflake.com/en/sql-reference/functions/div0
|
||||
def _div0_to_if(args: t.List) -> exp.Expression:
|
||||
def _div0_to_if(args: t.List) -> exp.If:
|
||||
cond = exp.EQ(this=seq_get(args, 1), expression=exp.Literal.number(0))
|
||||
true = exp.Literal.number(0)
|
||||
false = exp.Div(this=seq_get(args, 0), expression=seq_get(args, 1))
|
||||
|
|
@ -145,13 +145,13 @@ def _div0_to_if(args: t.List) -> exp.Expression:
|
|||
|
||||
|
||||
# https://docs.snowflake.com/en/sql-reference/functions/zeroifnull
|
||||
def _zeroifnull_to_if(args: t.List) -> exp.Expression:
|
||||
def _zeroifnull_to_if(args: t.List) -> exp.If:
|
||||
cond = exp.Is(this=seq_get(args, 0), expression=exp.Null())
|
||||
return exp.If(this=cond, true=exp.Literal.number(0), false=seq_get(args, 0))
|
||||
|
||||
|
||||
# https://docs.snowflake.com/en/sql-reference/functions/zeroifnull
|
||||
def _nullifzero_to_if(args: t.List) -> exp.Expression:
|
||||
def _nullifzero_to_if(args: t.List) -> exp.If:
|
||||
cond = exp.EQ(this=seq_get(args, 0), expression=exp.Literal.number(0))
|
||||
return exp.If(this=cond, true=exp.Null(), false=seq_get(args, 0))
|
||||
|
||||
|
|
@ -164,12 +164,21 @@ def _datatype_sql(self: generator.Generator, expression: exp.DataType) -> str:
|
|||
return self.datatype_sql(expression)
|
||||
|
||||
|
||||
def _parse_convert_timezone(args: t.List) -> exp.Expression:
|
||||
def _parse_convert_timezone(args: t.List) -> t.Union[exp.Anonymous, exp.AtTimeZone]:
|
||||
if len(args) == 3:
|
||||
return exp.Anonymous(this="CONVERT_TIMEZONE", expressions=args)
|
||||
return exp.AtTimeZone(this=seq_get(args, 1), zone=seq_get(args, 0))
|
||||
|
||||
|
||||
def _parse_regexp_replace(args: t.List) -> exp.RegexpReplace:
|
||||
regexp_replace = exp.RegexpReplace.from_arg_list(args)
|
||||
|
||||
if not regexp_replace.args.get("replacement"):
|
||||
regexp_replace.set("replacement", exp.Literal.string(""))
|
||||
|
||||
return regexp_replace
|
||||
|
||||
|
||||
class Snowflake(Dialect):
|
||||
# https://docs.snowflake.com/en/sql-reference/identifiers-syntax
|
||||
RESOLVES_IDENTIFIERS_AS_UPPERCASE = True
|
||||
|
|
@ -223,13 +232,14 @@ class Snowflake(Dialect):
|
|||
"IFF": exp.If.from_arg_list,
|
||||
"NULLIFZERO": _nullifzero_to_if,
|
||||
"OBJECT_CONSTRUCT": _parse_object_construct,
|
||||
"REGEXP_REPLACE": _parse_regexp_replace,
|
||||
"REGEXP_SUBSTR": exp.RegexpExtract.from_arg_list,
|
||||
"RLIKE": exp.RegexpLike.from_arg_list,
|
||||
"SQUARE": lambda args: exp.Pow(this=seq_get(args, 0), expression=exp.Literal.number(2)),
|
||||
"TIMEDIFF": _parse_datediff,
|
||||
"TIMESTAMPDIFF": _parse_datediff,
|
||||
"TO_ARRAY": exp.Array.from_arg_list,
|
||||
"TO_TIMESTAMP": _snowflake_to_timestamp,
|
||||
"TO_TIMESTAMP": _parse_to_timestamp,
|
||||
"TO_VARCHAR": exp.ToChar.from_arg_list,
|
||||
"ZEROIFNULL": _zeroifnull_to_if,
|
||||
}
|
||||
|
|
@ -242,7 +252,6 @@ class Snowflake(Dialect):
|
|||
|
||||
FUNC_TOKENS = {
|
||||
*parser.Parser.FUNC_TOKENS,
|
||||
TokenType.RLIKE,
|
||||
TokenType.TABLE,
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -2,9 +2,11 @@ from __future__ import annotations
|
|||
|
||||
import typing as t
|
||||
|
||||
from sqlglot import exp, parser, transforms
|
||||
from sqlglot import exp, transforms
|
||||
from sqlglot.dialects.dialect import (
|
||||
binary_from_function,
|
||||
create_with_partitions_sql,
|
||||
format_time_lambda,
|
||||
pivot_column_names,
|
||||
rename_func,
|
||||
trim_sql,
|
||||
|
|
@ -108,47 +110,36 @@ class Spark2(Hive):
|
|||
class Parser(Hive.Parser):
|
||||
FUNCTIONS = {
|
||||
**Hive.Parser.FUNCTIONS,
|
||||
"MAP_FROM_ARRAYS": exp.Map.from_arg_list,
|
||||
"TO_UNIX_TIMESTAMP": exp.StrToUnix.from_arg_list,
|
||||
"SHIFTLEFT": lambda args: exp.BitwiseLeftShift(
|
||||
this=seq_get(args, 0),
|
||||
expression=seq_get(args, 1),
|
||||
),
|
||||
"SHIFTRIGHT": lambda args: exp.BitwiseRightShift(
|
||||
this=seq_get(args, 0),
|
||||
expression=seq_get(args, 1),
|
||||
),
|
||||
"APPROX_PERCENTILE": exp.ApproxQuantile.from_arg_list,
|
||||
"IIF": exp.If.from_arg_list,
|
||||
"AGGREGATE": exp.Reduce.from_arg_list,
|
||||
"DAYOFWEEK": lambda args: exp.DayOfWeek(
|
||||
this=exp.TsOrDsToDate(this=seq_get(args, 0)),
|
||||
),
|
||||
"DAYOFMONTH": lambda args: exp.DayOfMonth(
|
||||
this=exp.TsOrDsToDate(this=seq_get(args, 0)),
|
||||
),
|
||||
"DAYOFYEAR": lambda args: exp.DayOfYear(
|
||||
this=exp.TsOrDsToDate(this=seq_get(args, 0)),
|
||||
),
|
||||
"WEEKOFYEAR": lambda args: exp.WeekOfYear(
|
||||
this=exp.TsOrDsToDate(this=seq_get(args, 0)),
|
||||
),
|
||||
"DATE_TRUNC": lambda args: exp.TimestampTrunc(
|
||||
this=seq_get(args, 1),
|
||||
unit=exp.var(seq_get(args, 0)),
|
||||
),
|
||||
"TRUNC": lambda args: exp.DateTrunc(unit=seq_get(args, 1), this=seq_get(args, 0)),
|
||||
"APPROX_PERCENTILE": exp.ApproxQuantile.from_arg_list,
|
||||
"BOOLEAN": _parse_as_cast("boolean"),
|
||||
"DATE": _parse_as_cast("date"),
|
||||
"DATE_TRUNC": lambda args: exp.TimestampTrunc(
|
||||
this=seq_get(args, 1), unit=exp.var(seq_get(args, 0))
|
||||
),
|
||||
"DAYOFMONTH": lambda args: exp.DayOfMonth(this=exp.TsOrDsToDate(this=seq_get(args, 0))),
|
||||
"DAYOFWEEK": lambda args: exp.DayOfWeek(this=exp.TsOrDsToDate(this=seq_get(args, 0))),
|
||||
"DAYOFYEAR": lambda args: exp.DayOfYear(this=exp.TsOrDsToDate(this=seq_get(args, 0))),
|
||||
"DOUBLE": _parse_as_cast("double"),
|
||||
"FLOAT": _parse_as_cast("float"),
|
||||
"IIF": exp.If.from_arg_list,
|
||||
"INT": _parse_as_cast("int"),
|
||||
"MAP_FROM_ARRAYS": exp.Map.from_arg_list,
|
||||
"RLIKE": exp.RegexpLike.from_arg_list,
|
||||
"SHIFTLEFT": binary_from_function(exp.BitwiseLeftShift),
|
||||
"SHIFTRIGHT": binary_from_function(exp.BitwiseRightShift),
|
||||
"STRING": _parse_as_cast("string"),
|
||||
"TIMESTAMP": _parse_as_cast("timestamp"),
|
||||
"TO_TIMESTAMP": lambda args: _parse_as_cast("timestamp")(args)
|
||||
if len(args) == 1
|
||||
else format_time_lambda(exp.StrToTime, "spark")(args),
|
||||
"TO_UNIX_TIMESTAMP": exp.StrToUnix.from_arg_list,
|
||||
"TRUNC": lambda args: exp.DateTrunc(unit=seq_get(args, 1), this=seq_get(args, 0)),
|
||||
"WEEKOFYEAR": lambda args: exp.WeekOfYear(this=exp.TsOrDsToDate(this=seq_get(args, 0))),
|
||||
}
|
||||
|
||||
FUNCTION_PARSERS = {
|
||||
**parser.Parser.FUNCTION_PARSERS,
|
||||
**Hive.Parser.FUNCTION_PARSERS,
|
||||
"BROADCAST": lambda self: self._parse_join_hint("BROADCAST"),
|
||||
"BROADCASTJOIN": lambda self: self._parse_join_hint("BROADCASTJOIN"),
|
||||
"MAPJOIN": lambda self: self._parse_join_hint("MAPJOIN"),
|
||||
|
|
@ -207,6 +198,13 @@ class Spark2(Hive):
|
|||
exp.Map: _map_sql,
|
||||
exp.Pivot: transforms.preprocess([_unqualify_pivot_columns]),
|
||||
exp.Reduce: rename_func("AGGREGATE"),
|
||||
exp.RegexpReplace: lambda self, e: self.func(
|
||||
"REGEXP_REPLACE",
|
||||
e.this,
|
||||
e.expression,
|
||||
e.args["replacement"],
|
||||
e.args.get("position"),
|
||||
),
|
||||
exp.StrToDate: _str_to_date,
|
||||
exp.StrToTime: lambda self, e: f"TO_TIMESTAMP({self.sql(e, 'this')}, {self.format_time(e)})",
|
||||
exp.TimestampTrunc: lambda self, e: self.func(
|
||||
|
|
@ -224,6 +222,7 @@ class Spark2(Hive):
|
|||
TRANSFORMS.pop(exp.ArraySort)
|
||||
TRANSFORMS.pop(exp.ILike)
|
||||
TRANSFORMS.pop(exp.Left)
|
||||
TRANSFORMS.pop(exp.MonthsBetween)
|
||||
TRANSFORMS.pop(exp.Right)
|
||||
|
||||
WRAP_DERIVED_VALUES = False
|
||||
|
|
|
|||
|
|
@ -20,6 +20,8 @@ class StarRocks(MySQL):
|
|||
}
|
||||
|
||||
class Generator(MySQL.Generator):
|
||||
CAST_MAPPING = {}
|
||||
|
||||
TYPE_MAPPING = {
|
||||
**MySQL.Generator.TYPE_MAPPING,
|
||||
exp.DataType.Type.TEXT: "STRING",
|
||||
|
|
|
|||
|
|
@ -138,7 +138,8 @@ def _format_sql(self: generator.Generator, expression: exp.NumberToStr | exp.Tim
|
|||
if isinstance(expression, exp.NumberToStr)
|
||||
else exp.Literal.string(
|
||||
format_time(
|
||||
expression.text("format"), t.cast(t.Dict[str, str], TSQL.INVERSE_TIME_MAPPING)
|
||||
expression.text("format"),
|
||||
t.cast(t.Dict[str, str], TSQL.INVERSE_TIME_MAPPING),
|
||||
)
|
||||
)
|
||||
)
|
||||
|
|
@ -314,7 +315,9 @@ class TSQL(Dialect):
|
|||
FUNCTIONS = {
|
||||
**parser.Parser.FUNCTIONS,
|
||||
"CHARINDEX": lambda args: exp.StrPosition(
|
||||
this=seq_get(args, 1), substr=seq_get(args, 0), position=seq_get(args, 2)
|
||||
this=seq_get(args, 1),
|
||||
substr=seq_get(args, 0),
|
||||
position=seq_get(args, 2),
|
||||
),
|
||||
"DATEADD": parse_date_delta(exp.DateAdd, unit_mapping=DATE_DELTA_INTERVAL),
|
||||
"DATEDIFF": parse_date_delta(exp.DateDiff, unit_mapping=DATE_DELTA_INTERVAL),
|
||||
|
|
@ -365,6 +368,55 @@ class TSQL(Dialect):
|
|||
|
||||
CONCAT_NULL_OUTPUTS_STRING = True
|
||||
|
||||
def _parse_commit_or_rollback(self) -> exp.Commit | exp.Rollback:
|
||||
"""Applies to SQL Server and Azure SQL Database
|
||||
COMMIT [ { TRAN | TRANSACTION }
|
||||
[ transaction_name | @tran_name_variable ] ]
|
||||
[ WITH ( DELAYED_DURABILITY = { OFF | ON } ) ]
|
||||
|
||||
ROLLBACK { TRAN | TRANSACTION }
|
||||
[ transaction_name | @tran_name_variable
|
||||
| savepoint_name | @savepoint_variable ]
|
||||
"""
|
||||
rollback = self._prev.token_type == TokenType.ROLLBACK
|
||||
|
||||
self._match_texts({"TRAN", "TRANSACTION"})
|
||||
this = self._parse_id_var()
|
||||
|
||||
if rollback:
|
||||
return self.expression(exp.Rollback, this=this)
|
||||
|
||||
durability = None
|
||||
if self._match_pair(TokenType.WITH, TokenType.L_PAREN):
|
||||
self._match_text_seq("DELAYED_DURABILITY")
|
||||
self._match(TokenType.EQ)
|
||||
|
||||
if self._match_text_seq("OFF"):
|
||||
durability = False
|
||||
else:
|
||||
self._match(TokenType.ON)
|
||||
durability = True
|
||||
|
||||
self._match_r_paren()
|
||||
|
||||
return self.expression(exp.Commit, this=this, durability=durability)
|
||||
|
||||
def _parse_transaction(self) -> exp.Transaction | exp.Command:
|
||||
"""Applies to SQL Server and Azure SQL Database
|
||||
BEGIN { TRAN | TRANSACTION }
|
||||
[ { transaction_name | @tran_name_variable }
|
||||
[ WITH MARK [ 'description' ] ]
|
||||
]
|
||||
"""
|
||||
if self._match_texts(("TRAN", "TRANSACTION")):
|
||||
transaction = self.expression(exp.Transaction, this=self._parse_id_var())
|
||||
if self._match_text_seq("WITH", "MARK"):
|
||||
transaction.set("mark", self._parse_string())
|
||||
|
||||
return transaction
|
||||
|
||||
return self._parse_as_command(self._prev)
|
||||
|
||||
def _parse_system_time(self) -> t.Optional[exp.Expression]:
|
||||
if not self._match_text_seq("FOR", "SYSTEM_TIME"):
|
||||
return None
|
||||
|
|
@ -496,7 +548,9 @@ class TSQL(Dialect):
|
|||
exp.Select: transforms.preprocess([transforms.eliminate_distinct_on]),
|
||||
exp.SHA: lambda self, e: self.func("HASHBYTES", exp.Literal.string("SHA1"), e.this),
|
||||
exp.SHA2: lambda self, e: self.func(
|
||||
"HASHBYTES", exp.Literal.string(f"SHA2_{e.args.get('length', 256)}"), e.this
|
||||
"HASHBYTES",
|
||||
exp.Literal.string(f"SHA2_{e.args.get('length', 256)}"),
|
||||
e.this,
|
||||
),
|
||||
exp.TimeToStr: _format_sql,
|
||||
}
|
||||
|
|
@ -539,3 +593,26 @@ class TSQL(Dialect):
|
|||
into = self.sql(expression, "into")
|
||||
into = self.seg(f"INTO {into}") if into else ""
|
||||
return f"{self.seg('OUTPUT')} {self.expressions(expression, flat=True)}{into}"
|
||||
|
||||
def transaction_sql(self, expression: exp.Transaction) -> str:
|
||||
this = self.sql(expression, "this")
|
||||
this = f" {this}" if this else ""
|
||||
mark = self.sql(expression, "mark")
|
||||
mark = f" WITH MARK {mark}" if mark else ""
|
||||
return f"BEGIN TRANSACTION{this}{mark}"
|
||||
|
||||
def commit_sql(self, expression: exp.Commit) -> str:
|
||||
this = self.sql(expression, "this")
|
||||
this = f" {this}" if this else ""
|
||||
durability = expression.args.get("durability")
|
||||
durability = (
|
||||
f" WITH (DELAYED_DURABILITY = {'ON' if durability else 'OFF'})"
|
||||
if durability is not None
|
||||
else ""
|
||||
)
|
||||
return f"COMMIT TRANSACTION{this}{durability}"
|
||||
|
||||
def rollback_sql(self, expression: exp.Rollback) -> str:
|
||||
this = self.sql(expression, "this")
|
||||
this = f" {this}" if this else ""
|
||||
return f"ROLLBACK TRANSACTION{this}"
|
||||
|
|
|
|||
|
|
@ -759,12 +759,24 @@ class Condition(Expression):
|
|||
)
|
||||
|
||||
def isin(
|
||||
self, *expressions: t.Any, query: t.Optional[ExpOrStr] = None, copy: bool = True, **opts
|
||||
self,
|
||||
*expressions: t.Any,
|
||||
query: t.Optional[ExpOrStr] = None,
|
||||
unnest: t.Optional[ExpOrStr] | t.Collection[ExpOrStr] = None,
|
||||
copy: bool = True,
|
||||
**opts,
|
||||
) -> In:
|
||||
return In(
|
||||
this=_maybe_copy(self, copy),
|
||||
expressions=[convert(e, copy=copy) for e in expressions],
|
||||
query=maybe_parse(query, copy=copy, **opts) if query else None,
|
||||
unnest=Unnest(
|
||||
expressions=[
|
||||
maybe_parse(t.cast(ExpOrStr, e), copy=copy, **opts) for e in ensure_list(unnest)
|
||||
]
|
||||
)
|
||||
if unnest
|
||||
else None,
|
||||
)
|
||||
|
||||
def between(self, low: t.Any, high: t.Any, copy: bool = True, **opts) -> Between:
|
||||
|
|
@ -2019,7 +2031,20 @@ class RowFormatDelimitedProperty(Property):
|
|||
|
||||
|
||||
class RowFormatSerdeProperty(Property):
|
||||
arg_types = {"this": True}
|
||||
arg_types = {"this": True, "serde_properties": False}
|
||||
|
||||
|
||||
# https://spark.apache.org/docs/3.1.2/sql-ref-syntax-qry-select-transform.html
|
||||
class QueryTransform(Expression):
|
||||
arg_types = {
|
||||
"expressions": True,
|
||||
"command_script": True,
|
||||
"schema": False,
|
||||
"row_format_before": False,
|
||||
"record_writer": False,
|
||||
"row_format_after": False,
|
||||
"record_reader": False,
|
||||
}
|
||||
|
||||
|
||||
class SchemaCommentProperty(Property):
|
||||
|
|
@ -2149,12 +2174,24 @@ class Tuple(Expression):
|
|||
arg_types = {"expressions": False}
|
||||
|
||||
def isin(
|
||||
self, *expressions: t.Any, query: t.Optional[ExpOrStr] = None, copy: bool = True, **opts
|
||||
self,
|
||||
*expressions: t.Any,
|
||||
query: t.Optional[ExpOrStr] = None,
|
||||
unnest: t.Optional[ExpOrStr] | t.Collection[ExpOrStr] = None,
|
||||
copy: bool = True,
|
||||
**opts,
|
||||
) -> In:
|
||||
return In(
|
||||
this=_maybe_copy(self, copy),
|
||||
expressions=[convert(e, copy=copy) for e in expressions],
|
||||
query=maybe_parse(query, copy=copy, **opts) if query else None,
|
||||
unnest=Unnest(
|
||||
expressions=[
|
||||
maybe_parse(t.cast(ExpOrStr, e), copy=copy, **opts) for e in ensure_list(unnest)
|
||||
]
|
||||
)
|
||||
if unnest
|
||||
else None,
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -3478,15 +3515,15 @@ class Command(Expression):
|
|||
|
||||
|
||||
class Transaction(Expression):
|
||||
arg_types = {"this": False, "modes": False}
|
||||
arg_types = {"this": False, "modes": False, "mark": False}
|
||||
|
||||
|
||||
class Commit(Expression):
|
||||
arg_types = {"chain": False}
|
||||
arg_types = {"chain": False, "this": False, "durability": False}
|
||||
|
||||
|
||||
class Rollback(Expression):
|
||||
arg_types = {"savepoint": False}
|
||||
arg_types = {"savepoint": False, "this": False}
|
||||
|
||||
|
||||
class AlterTable(Expression):
|
||||
|
|
@ -3530,10 +3567,6 @@ class Or(Connector):
|
|||
pass
|
||||
|
||||
|
||||
class Xor(Connector):
|
||||
pass
|
||||
|
||||
|
||||
class BitwiseAnd(Binary):
|
||||
pass
|
||||
|
||||
|
|
@ -3856,6 +3889,11 @@ class Abs(Func):
|
|||
pass
|
||||
|
||||
|
||||
# https://spark.apache.org/docs/latest/api/sql/index.html#transform
|
||||
class Transform(Func):
|
||||
arg_types = {"this": True, "expression": True}
|
||||
|
||||
|
||||
class Anonymous(Func):
|
||||
arg_types = {"this": True, "expressions": False}
|
||||
is_var_len_args = True
|
||||
|
|
@ -4098,6 +4136,10 @@ class WeekOfYear(Func):
|
|||
_sql_names = ["WEEK_OF_YEAR", "WEEKOFYEAR"]
|
||||
|
||||
|
||||
class MonthsBetween(Func):
|
||||
arg_types = {"this": True, "expression": True, "roundoff": False}
|
||||
|
||||
|
||||
class LastDateOfMonth(Func):
|
||||
pass
|
||||
|
||||
|
|
@ -4209,6 +4251,10 @@ class Hex(Func):
|
|||
pass
|
||||
|
||||
|
||||
class Xor(Connector, Func):
|
||||
arg_types = {"this": False, "expression": False, "expressions": False}
|
||||
|
||||
|
||||
class If(Func):
|
||||
arg_types = {"this": True, "true": True, "false": False}
|
||||
|
||||
|
|
@ -4431,7 +4477,18 @@ class RegexpExtract(Func):
|
|||
}
|
||||
|
||||
|
||||
class RegexpLike(Func):
|
||||
class RegexpReplace(Func):
|
||||
arg_types = {
|
||||
"this": True,
|
||||
"expression": True,
|
||||
"replacement": True,
|
||||
"position": False,
|
||||
"occurrence": False,
|
||||
"parameters": False,
|
||||
}
|
||||
|
||||
|
||||
class RegexpLike(Binary, Func):
|
||||
arg_types = {"this": True, "expression": True, "flag": False}
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -164,6 +164,11 @@ class Generator:
|
|||
# https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax
|
||||
SELECT_KINDS: t.Tuple[str, ...] = ("STRUCT", "VALUE")
|
||||
|
||||
# Whether or not VALUES statements can be used as derived tables.
|
||||
# MySQL 5 and Redshift do not allow this, so when False, it will convert
|
||||
# SELECT * VALUES into SELECT UNION
|
||||
VALUES_AS_TABLE = True
|
||||
|
||||
TYPE_MAPPING = {
|
||||
exp.DataType.Type.NCHAR: "CHAR",
|
||||
exp.DataType.Type.NVARCHAR: "VARCHAR",
|
||||
|
|
@ -260,8 +265,9 @@ class Generator:
|
|||
|
||||
# Expressions whose comments are separated from them for better formatting
|
||||
WITH_SEPARATED_COMMENTS: t.Tuple[t.Type[exp.Expression], ...] = (
|
||||
exp.Select,
|
||||
exp.Drop,
|
||||
exp.From,
|
||||
exp.Select,
|
||||
exp.Where,
|
||||
exp.With,
|
||||
)
|
||||
|
|
@ -818,7 +824,11 @@ class Generator:
|
|||
|
||||
def datatype_sql(self, expression: exp.DataType) -> str:
|
||||
type_value = expression.this
|
||||
type_sql = self.TYPE_MAPPING.get(type_value, type_value.value)
|
||||
type_sql = (
|
||||
self.TYPE_MAPPING.get(type_value, type_value.value)
|
||||
if isinstance(type_value, exp.DataType.Type)
|
||||
else type_value
|
||||
)
|
||||
nested = ""
|
||||
interior = self.expressions(expression, flat=True)
|
||||
values = ""
|
||||
|
|
@ -1307,15 +1317,45 @@ class Generator:
|
|||
return self.prepend_ctes(expression, sql)
|
||||
|
||||
def values_sql(self, expression: exp.Values) -> str:
|
||||
args = self.expressions(expression)
|
||||
alias = self.sql(expression, "alias")
|
||||
values = f"VALUES{self.seg('')}{args}"
|
||||
values = (
|
||||
f"({values})"
|
||||
if self.WRAP_DERIVED_VALUES and (alias or isinstance(expression.parent, exp.From))
|
||||
else values
|
||||
)
|
||||
return f"{values} AS {alias}" if alias else values
|
||||
# The VALUES clause is still valid in an `INSERT INTO ..` statement, for example
|
||||
if self.VALUES_AS_TABLE or not expression.find_ancestor(exp.From, exp.Join):
|
||||
args = self.expressions(expression)
|
||||
alias = self.sql(expression, "alias")
|
||||
values = f"VALUES{self.seg('')}{args}"
|
||||
values = (
|
||||
f"({values})"
|
||||
if self.WRAP_DERIVED_VALUES and (alias or isinstance(expression.parent, exp.From))
|
||||
else values
|
||||
)
|
||||
return f"{values} AS {alias}" if alias else values
|
||||
|
||||
# Converts `VALUES...` expression into a series of select unions.
|
||||
# Note: If you have a lot of unions then this will result in a large number of recursive statements to
|
||||
# evaluate the expression. You may need to increase `sys.setrecursionlimit` to run and it can also be
|
||||
# very slow.
|
||||
expression = expression.copy()
|
||||
column_names = expression.alias and expression.args["alias"].columns
|
||||
|
||||
selects = []
|
||||
|
||||
for i, tup in enumerate(expression.expressions):
|
||||
row = tup.expressions
|
||||
|
||||
if i == 0 and column_names:
|
||||
row = [
|
||||
exp.alias_(value, column_name) for value, column_name in zip(row, column_names)
|
||||
]
|
||||
|
||||
selects.append(exp.Select(expressions=row))
|
||||
|
||||
subquery_expression: exp.Select | exp.Union = selects[0]
|
||||
if len(selects) > 1:
|
||||
for select in selects[1:]:
|
||||
subquery_expression = exp.union(
|
||||
subquery_expression, select, distinct=False, copy=False
|
||||
)
|
||||
|
||||
return self.subquery_sql(subquery_expression.subquery(expression.alias, copy=False))
|
||||
|
||||
def var_sql(self, expression: exp.Var) -> str:
|
||||
return self.sql(expression, "this")
|
||||
|
|
@ -2043,7 +2083,7 @@ class Generator:
|
|||
def and_sql(self, expression: exp.And) -> str:
|
||||
return self.connector_sql(expression, "AND")
|
||||
|
||||
def xor_sql(self, expression: exp.And) -> str:
|
||||
def xor_sql(self, expression: exp.Xor) -> str:
|
||||
return self.connector_sql(expression, "XOR")
|
||||
|
||||
def connector_sql(self, expression: exp.Connector, op: str) -> str:
|
||||
|
|
@ -2507,6 +2547,21 @@ class Generator:
|
|||
|
||||
return self.func("ANY_VALUE", this)
|
||||
|
||||
def querytransform_sql(self, expression: exp.QueryTransform) -> str:
|
||||
transform = self.func("TRANSFORM", *expression.expressions)
|
||||
row_format_before = self.sql(expression, "row_format_before")
|
||||
row_format_before = f" {row_format_before}" if row_format_before else ""
|
||||
record_writer = self.sql(expression, "record_writer")
|
||||
record_writer = f" RECORDWRITER {record_writer}" if record_writer else ""
|
||||
using = f" USING {self.sql(expression, 'command_script')}"
|
||||
schema = self.sql(expression, "schema")
|
||||
schema = f" AS {schema}" if schema else ""
|
||||
row_format_after = self.sql(expression, "row_format_after")
|
||||
row_format_after = f" {row_format_after}" if row_format_after else ""
|
||||
record_reader = self.sql(expression, "record_reader")
|
||||
record_reader = f" RECORDREADER {record_reader}" if record_reader else ""
|
||||
return f"{transform}{row_format_before}{record_writer}{using}{schema}{row_format_after}{record_reader}"
|
||||
|
||||
|
||||
def cached_generator(
|
||||
cache: t.Optional[t.Dict[int, str]] = None
|
||||
|
|
|
|||
|
|
@ -79,7 +79,7 @@ def lineage(
|
|||
raise SqlglotError("Cannot build lineage, sql must be SELECT")
|
||||
|
||||
def to_node(
|
||||
column_name: str,
|
||||
column: str | int,
|
||||
scope: Scope,
|
||||
scope_name: t.Optional[str] = None,
|
||||
upstream: t.Optional[Node] = None,
|
||||
|
|
@ -90,26 +90,38 @@ def lineage(
|
|||
for dt in scope.derived_tables
|
||||
if dt.comments and dt.comments[0].startswith("source: ")
|
||||
}
|
||||
if isinstance(scope.expression, exp.Union):
|
||||
for scope in scope.union_scopes:
|
||||
node = to_node(
|
||||
column_name,
|
||||
scope=scope,
|
||||
scope_name=scope_name,
|
||||
upstream=upstream,
|
||||
alias=aliases.get(scope_name),
|
||||
)
|
||||
return node
|
||||
|
||||
# Find the specific select clause that is the source of the column we want.
|
||||
# This can either be a specific, named select or a generic `*` clause.
|
||||
select = next(
|
||||
(select for select in scope.expression.selects if select.alias_or_name == column_name),
|
||||
exp.Star() if scope.expression.is_star else None,
|
||||
select = (
|
||||
scope.expression.selects[column]
|
||||
if isinstance(column, int)
|
||||
else next(
|
||||
(select for select in scope.expression.selects if select.alias_or_name == column),
|
||||
exp.Star() if scope.expression.is_star else None,
|
||||
)
|
||||
)
|
||||
|
||||
if not select:
|
||||
raise ValueError(f"Could not find {column_name} in {scope.expression}")
|
||||
raise ValueError(f"Could not find {column} in {scope.expression}")
|
||||
|
||||
if isinstance(scope.expression, exp.Union):
|
||||
upstream = upstream or Node(name="UNION", source=scope.expression, expression=select)
|
||||
|
||||
index = (
|
||||
column
|
||||
if isinstance(column, int)
|
||||
else next(
|
||||
i
|
||||
for i, select in enumerate(scope.expression.selects)
|
||||
if select.alias_or_name == column
|
||||
)
|
||||
)
|
||||
|
||||
for s in scope.union_scopes:
|
||||
to_node(index, scope=s, upstream=upstream)
|
||||
|
||||
return upstream
|
||||
|
||||
if isinstance(scope.expression, exp.Select):
|
||||
# For better ergonomics in our node labels, replace the full select with
|
||||
|
|
@ -122,7 +134,7 @@ def lineage(
|
|||
|
||||
# Create the node for this step in the lineage chain, and attach it to the previous one.
|
||||
node = Node(
|
||||
name=f"{scope_name}.{column_name}" if scope_name else column_name,
|
||||
name=f"{scope_name}.{column}" if scope_name else str(column),
|
||||
source=source,
|
||||
expression=select,
|
||||
alias=alias or "",
|
||||
|
|
|
|||
|
|
@ -144,8 +144,9 @@ def _eliminate_derived_table(scope, existing_ctes, taken):
|
|||
name, cte = _new_cte(scope, existing_ctes, taken)
|
||||
|
||||
table = exp.alias_(exp.table_(name), alias=parent.alias or name)
|
||||
parent.replace(table)
|
||||
table.set("joins", parent.args.get("joins"))
|
||||
|
||||
parent.replace(table)
|
||||
return cte
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -176,6 +176,7 @@ def _mergeable(outer_scope, inner_scope, leave_tables_isolated, from_or_join):
|
|||
|
||||
return (
|
||||
isinstance(outer_scope.expression, exp.Select)
|
||||
and not outer_scope.expression.is_star
|
||||
and isinstance(inner_select, exp.Select)
|
||||
and not any(inner_select.args.get(arg) for arg in UNMERGABLE_ARGS)
|
||||
and inner_select.args.get("from")
|
||||
|
|
@ -242,6 +243,7 @@ def _merge_from(outer_scope, inner_scope, node_to_replace, alias):
|
|||
alias (str)
|
||||
"""
|
||||
new_subquery = inner_scope.expression.args["from"].this
|
||||
new_subquery.set("joins", node_to_replace.args.get("joins"))
|
||||
node_to_replace.replace(new_subquery)
|
||||
for join_hint in outer_scope.join_hints:
|
||||
tables = join_hint.find_all(exp.Table)
|
||||
|
|
|
|||
|
|
@ -61,6 +61,9 @@ def pushdown_projections(expression, schema=None, remove_unused_selections=True)
|
|||
if remove_unused_selections:
|
||||
_remove_unused_selections(scope, parent_selections, schema)
|
||||
|
||||
if scope.expression.is_star:
|
||||
continue
|
||||
|
||||
# Group columns by source name
|
||||
selects = defaultdict(set)
|
||||
for col in scope.columns:
|
||||
|
|
|
|||
|
|
@ -29,12 +29,13 @@ def qualify_columns(
|
|||
'SELECT tbl.col AS col FROM tbl'
|
||||
|
||||
Args:
|
||||
expression: expression to qualify
|
||||
schema: Database schema
|
||||
expand_alias_refs: whether or not to expand references to aliases
|
||||
infer_schema: whether or not to infer the schema if missing
|
||||
expression: Expression to qualify.
|
||||
schema: Database schema.
|
||||
expand_alias_refs: Whether or not to expand references to aliases.
|
||||
infer_schema: Whether or not to infer the schema if missing.
|
||||
|
||||
Returns:
|
||||
sqlglot.Expression: qualified expression
|
||||
The qualified expression.
|
||||
"""
|
||||
schema = ensure_schema(schema)
|
||||
infer_schema = schema.empty if infer_schema is None else infer_schema
|
||||
|
|
@ -410,7 +411,9 @@ def _expand_stars(
|
|||
else:
|
||||
return
|
||||
|
||||
scope.expression.set("expressions", new_selections)
|
||||
# Ensures we don't overwrite the initial selections with an empty list
|
||||
if new_selections:
|
||||
scope.expression.set("expressions", new_selections)
|
||||
|
||||
|
||||
def _add_except_columns(
|
||||
|
|
|
|||
|
|
@ -124,8 +124,8 @@ class Scope:
|
|||
self._ctes.append(node)
|
||||
elif (
|
||||
isinstance(node, exp.Subquery)
|
||||
and isinstance(parent, (exp.From, exp.Join))
|
||||
and _is_subquery_scope(node)
|
||||
and isinstance(parent, (exp.From, exp.Join, exp.Subquery))
|
||||
and _is_derived_table(node)
|
||||
):
|
||||
self._derived_tables.append(node)
|
||||
elif isinstance(node, exp.Subqueryable):
|
||||
|
|
@ -610,13 +610,13 @@ def _traverse_ctes(scope):
|
|||
scope.sources.update(sources)
|
||||
|
||||
|
||||
def _is_subquery_scope(expression: exp.Subquery) -> bool:
|
||||
def _is_derived_table(expression: exp.Subquery) -> bool:
|
||||
"""
|
||||
We represent (tbl1 JOIN tbl2) as a Subquery, but it's not really a new scope.
|
||||
If an alias is present, it shadows all names under the Subquery, so that's an
|
||||
exception to this rule.
|
||||
We represent (tbl1 JOIN tbl2) as a Subquery, but it's not really a "derived table",
|
||||
as it doesn't introduce a new scope. If an alias is present, it shadows all names
|
||||
under the Subquery, so that's one exception to this rule.
|
||||
"""
|
||||
return bool(not isinstance(expression.unnest(), exp.Table) or expression.alias)
|
||||
return bool(expression.alias or isinstance(expression.this, exp.Subqueryable))
|
||||
|
||||
|
||||
def _traverse_tables(scope):
|
||||
|
|
@ -654,7 +654,10 @@ def _traverse_tables(scope):
|
|||
else:
|
||||
sources[source_name] = expression
|
||||
|
||||
expressions.extend(join.this for join in expression.args.get("joins") or [])
|
||||
# Make sure to not include the joins twice
|
||||
if expression is not scope.expression:
|
||||
expressions.extend(join.this for join in expression.args.get("joins") or [])
|
||||
|
||||
continue
|
||||
|
||||
if not isinstance(expression, exp.DerivedTable):
|
||||
|
|
@ -664,10 +667,11 @@ def _traverse_tables(scope):
|
|||
lateral_sources = sources
|
||||
scope_type = ScopeType.UDTF
|
||||
scopes = scope.udtf_scopes
|
||||
elif _is_subquery_scope(expression):
|
||||
elif _is_derived_table(expression):
|
||||
lateral_sources = None
|
||||
scope_type = ScopeType.DERIVED_TABLE
|
||||
scopes = scope.derived_table_scopes
|
||||
expressions.extend(join.this for join in expression.args.get("joins") or [])
|
||||
else:
|
||||
# Makes sure we check for possible sources in nested table constructs
|
||||
expressions.append(expression.this)
|
||||
|
|
@ -735,10 +739,16 @@ def walk_in_scope(expression, bfs=True):
|
|||
isinstance(node, exp.CTE)
|
||||
or (
|
||||
isinstance(node, exp.Subquery)
|
||||
and isinstance(parent, (exp.From, exp.Join))
|
||||
and _is_subquery_scope(node)
|
||||
and isinstance(parent, (exp.From, exp.Join, exp.Subquery))
|
||||
and _is_derived_table(node)
|
||||
)
|
||||
or isinstance(node, exp.UDTF)
|
||||
or isinstance(node, exp.Subqueryable)
|
||||
):
|
||||
prune = True
|
||||
|
||||
if isinstance(node, (exp.Subquery, exp.UDTF)):
|
||||
# The following args are not actually in the inner scope, so we should visit them
|
||||
for key in ("joins", "laterals", "pivots"):
|
||||
for arg in node.args.get(key) or []:
|
||||
yield from walk_in_scope(arg, bfs=bfs)
|
||||
|
|
|
|||
|
|
@ -327,6 +327,7 @@ class Parser(metaclass=_Parser):
|
|||
TokenType.PRIMARY_KEY,
|
||||
TokenType.RANGE,
|
||||
TokenType.REPLACE,
|
||||
TokenType.RLIKE,
|
||||
TokenType.ROW,
|
||||
TokenType.UNNEST,
|
||||
TokenType.VAR,
|
||||
|
|
@ -338,6 +339,7 @@ class Parser(metaclass=_Parser):
|
|||
TokenType.TIMESTAMP,
|
||||
TokenType.TIMESTAMPTZ,
|
||||
TokenType.WINDOW,
|
||||
TokenType.XOR,
|
||||
*TYPE_TOKENS,
|
||||
*SUBQUERY_PREDICATES,
|
||||
}
|
||||
|
|
@ -505,7 +507,6 @@ class Parser(metaclass=_Parser):
|
|||
TokenType.DESC: lambda self: self._parse_describe(),
|
||||
TokenType.DESCRIBE: lambda self: self._parse_describe(),
|
||||
TokenType.DROP: lambda self: self._parse_drop(),
|
||||
TokenType.END: lambda self: self._parse_commit_or_rollback(),
|
||||
TokenType.FROM: lambda self: exp.select("*").from_(
|
||||
t.cast(exp.From, self._parse_from(skip_from_token=True))
|
||||
),
|
||||
|
|
@ -716,7 +717,7 @@ class Parser(metaclass=_Parser):
|
|||
|
||||
FUNCTIONS_WITH_ALIASED_ARGS = {"STRUCT"}
|
||||
|
||||
FUNCTION_PARSERS: t.Dict[str, t.Callable] = {
|
||||
FUNCTION_PARSERS = {
|
||||
"ANY_VALUE": lambda self: self._parse_any_value(),
|
||||
"CAST": lambda self: self._parse_cast(self.STRICT_CAST),
|
||||
"CONCAT": lambda self: self._parse_concat(),
|
||||
|
|
@ -1144,6 +1145,7 @@ class Parser(metaclass=_Parser):
|
|||
|
||||
return self.expression(
|
||||
exp.Drop,
|
||||
comments=start.comments,
|
||||
exists=self._parse_exists(),
|
||||
this=self._parse_table(schema=True),
|
||||
kind=kind,
|
||||
|
|
@ -1233,11 +1235,14 @@ class Parser(metaclass=_Parser):
|
|||
expression = self._parse_ddl_select()
|
||||
|
||||
if create_token.token_type == TokenType.TABLE:
|
||||
# exp.Properties.Location.POST_EXPRESSION
|
||||
extend_props(self._parse_properties())
|
||||
|
||||
indexes = []
|
||||
while True:
|
||||
index = self._parse_index()
|
||||
|
||||
# exp.Properties.Location.POST_EXPRESSION and POST_INDEX
|
||||
# exp.Properties.Location.POST_INDEX
|
||||
extend_props(self._parse_properties())
|
||||
|
||||
if not index:
|
||||
|
|
@ -1384,7 +1389,6 @@ class Parser(metaclass=_Parser):
|
|||
def _parse_with_property(
|
||||
self,
|
||||
) -> t.Optional[exp.Expression] | t.List[t.Optional[exp.Expression]]:
|
||||
self._match(TokenType.WITH)
|
||||
if self._match(TokenType.L_PAREN, advance=False):
|
||||
return self._parse_wrapped_csv(self._parse_property)
|
||||
|
||||
|
|
@ -1781,7 +1785,17 @@ class Parser(metaclass=_Parser):
|
|||
return None
|
||||
|
||||
if self._match_text_seq("SERDE"):
|
||||
return self.expression(exp.RowFormatSerdeProperty, this=self._parse_string())
|
||||
this = self._parse_string()
|
||||
|
||||
serde_properties = None
|
||||
if self._match(TokenType.SERDE_PROPERTIES):
|
||||
serde_properties = self.expression(
|
||||
exp.SerdeProperties, expressions=self._parse_wrapped_csv(self._parse_property)
|
||||
)
|
||||
|
||||
return self.expression(
|
||||
exp.RowFormatSerdeProperty, this=this, serde_properties=serde_properties
|
||||
)
|
||||
|
||||
self._match_text_seq("DELIMITED")
|
||||
|
||||
|
|
@ -2251,7 +2265,9 @@ class Parser(metaclass=_Parser):
|
|||
self._match_set(self.JOIN_KINDS) and self._prev,
|
||||
)
|
||||
|
||||
def _parse_join(self, skip_join_token: bool = False) -> t.Optional[exp.Join]:
|
||||
def _parse_join(
|
||||
self, skip_join_token: bool = False, parse_bracket: bool = False
|
||||
) -> t.Optional[exp.Join]:
|
||||
if self._match(TokenType.COMMA):
|
||||
return self.expression(exp.Join, this=self._parse_table())
|
||||
|
||||
|
|
@ -2275,7 +2291,7 @@ class Parser(metaclass=_Parser):
|
|||
if outer_apply:
|
||||
side = Token(TokenType.LEFT, "LEFT")
|
||||
|
||||
kwargs: t.Dict[str, t.Any] = {"this": self._parse_table()}
|
||||
kwargs: t.Dict[str, t.Any] = {"this": self._parse_table(parse_bracket=parse_bracket)}
|
||||
|
||||
if method:
|
||||
kwargs["method"] = method.text
|
||||
|
|
@ -2411,6 +2427,7 @@ class Parser(metaclass=_Parser):
|
|||
schema: bool = False,
|
||||
joins: bool = False,
|
||||
alias_tokens: t.Optional[t.Collection[TokenType]] = None,
|
||||
parse_bracket: bool = False,
|
||||
) -> t.Optional[exp.Expression]:
|
||||
lateral = self._parse_lateral()
|
||||
if lateral:
|
||||
|
|
@ -2430,7 +2447,9 @@ class Parser(metaclass=_Parser):
|
|||
subquery.set("pivots", self._parse_pivots())
|
||||
return subquery
|
||||
|
||||
this: exp.Expression = self._parse_table_parts(schema=schema)
|
||||
bracket = parse_bracket and self._parse_bracket(None)
|
||||
bracket = self.expression(exp.Table, this=bracket) if bracket else None
|
||||
this: exp.Expression = bracket or self._parse_table_parts(schema=schema)
|
||||
|
||||
if schema:
|
||||
return self._parse_schema(this=this)
|
||||
|
|
@ -2758,8 +2777,15 @@ class Parser(metaclass=_Parser):
|
|||
self, this: t.Optional[exp.Expression] = None, top: bool = False
|
||||
) -> t.Optional[exp.Expression]:
|
||||
if self._match(TokenType.TOP if top else TokenType.LIMIT):
|
||||
limit_paren = self._match(TokenType.L_PAREN)
|
||||
expression = self._parse_number() if top else self._parse_term()
|
||||
comments = self._prev_comments
|
||||
if top:
|
||||
limit_paren = self._match(TokenType.L_PAREN)
|
||||
expression = self._parse_number()
|
||||
|
||||
if limit_paren:
|
||||
self._match_r_paren()
|
||||
else:
|
||||
expression = self._parse_term()
|
||||
|
||||
if self._match(TokenType.COMMA):
|
||||
offset = expression
|
||||
|
|
@ -2767,10 +2793,9 @@ class Parser(metaclass=_Parser):
|
|||
else:
|
||||
offset = None
|
||||
|
||||
limit_exp = self.expression(exp.Limit, this=this, expression=expression, offset=offset)
|
||||
|
||||
if limit_paren:
|
||||
self._match_r_paren()
|
||||
limit_exp = self.expression(
|
||||
exp.Limit, this=this, expression=expression, offset=offset, comments=comments
|
||||
)
|
||||
|
||||
return limit_exp
|
||||
|
||||
|
|
@ -2803,7 +2828,7 @@ class Parser(metaclass=_Parser):
|
|||
if not self._match(TokenType.OFFSET):
|
||||
return this
|
||||
|
||||
count = self._parse_number()
|
||||
count = self._parse_term()
|
||||
self._match_set((TokenType.ROW, TokenType.ROWS))
|
||||
return self.expression(exp.Offset, this=this, expression=count)
|
||||
|
||||
|
|
@ -3320,7 +3345,7 @@ class Parser(metaclass=_Parser):
|
|||
else:
|
||||
this = self.expression(exp.Anonymous, this=this, expressions=args)
|
||||
|
||||
self._match_r_paren(this)
|
||||
self._match(TokenType.R_PAREN, expression=this)
|
||||
return self._parse_window(this)
|
||||
|
||||
def _parse_function_parameter(self) -> t.Optional[exp.Expression]:
|
||||
|
|
@ -4076,7 +4101,10 @@ class Parser(metaclass=_Parser):
|
|||
self, this: t.Optional[exp.Expression], alias: bool = False
|
||||
) -> t.Optional[exp.Expression]:
|
||||
if self._match_pair(TokenType.FILTER, TokenType.L_PAREN):
|
||||
this = self.expression(exp.Filter, this=this, expression=self._parse_where())
|
||||
self._match(TokenType.WHERE)
|
||||
this = self.expression(
|
||||
exp.Filter, this=this, expression=self._parse_where(skip_where_token=True)
|
||||
)
|
||||
self._match_r_paren()
|
||||
|
||||
# T-SQL allows the OVER (...) syntax after WITHIN GROUP.
|
||||
|
|
@ -4351,7 +4379,7 @@ class Parser(metaclass=_Parser):
|
|||
self._parse_set_operations(self._parse_select(nested=True, parse_subquery_alias=False))
|
||||
)
|
||||
|
||||
def _parse_transaction(self) -> exp.Transaction:
|
||||
def _parse_transaction(self) -> exp.Transaction | exp.Command:
|
||||
this = None
|
||||
if self._match_texts(self.TRANSACTION_KIND):
|
||||
this = self._prev.text
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ from __future__ import annotations
|
|||
import typing as t
|
||||
from enum import auto
|
||||
|
||||
from sqlglot.errors import TokenError
|
||||
from sqlglot.helper import AutoName
|
||||
from sqlglot.trie import TrieResult, in_trie, new_trie
|
||||
|
||||
|
|
@ -800,7 +801,7 @@ class Tokenizer(metaclass=_Tokenizer):
|
|||
start = max(self._current - 50, 0)
|
||||
end = min(self._current + 50, self.size - 1)
|
||||
context = self.sql[start:end]
|
||||
raise ValueError(f"Error tokenizing '{context}'") from e
|
||||
raise TokenError(f"Error tokenizing '{context}'") from e
|
||||
|
||||
return self.tokens
|
||||
|
||||
|
|
@ -1097,7 +1098,7 @@ class Tokenizer(metaclass=_Tokenizer):
|
|||
try:
|
||||
int(text, base)
|
||||
except:
|
||||
raise RuntimeError(
|
||||
raise TokenError(
|
||||
f"Numeric string contains invalid characters from {self._line}:{self._start}"
|
||||
)
|
||||
else:
|
||||
|
|
@ -1140,7 +1141,7 @@ class Tokenizer(metaclass=_Tokenizer):
|
|||
if self._current + 1 < self.size:
|
||||
self._advance(2)
|
||||
else:
|
||||
raise RuntimeError(f"Missing {delimiter} from {self._line}:{self._current}")
|
||||
raise TokenError(f"Missing {delimiter} from {self._line}:{self._current}")
|
||||
else:
|
||||
if self._chars(delim_size) == delimiter:
|
||||
if delim_size > 1:
|
||||
|
|
@ -1148,7 +1149,7 @@ class Tokenizer(metaclass=_Tokenizer):
|
|||
break
|
||||
|
||||
if self._end:
|
||||
raise RuntimeError(f"Missing {delimiter} from {self._line}:{self._start}")
|
||||
raise TokenError(f"Missing {delimiter} from {self._line}:{self._start}")
|
||||
|
||||
current = self._current - 1
|
||||
self._advance(alnum=True)
|
||||
|
|
|
|||
|
|
@ -851,9 +851,9 @@ class TestFunctions(unittest.TestCase):
|
|||
|
||||
def test_to_timestamp(self):
|
||||
col_str = SF.to_timestamp("cola")
|
||||
self.assertEqual("TO_TIMESTAMP(cola)", col_str.sql())
|
||||
self.assertEqual("CAST(cola AS TIMESTAMP)", col_str.sql())
|
||||
col = SF.to_timestamp(SF.col("cola"))
|
||||
self.assertEqual("TO_TIMESTAMP(cola)", col.sql())
|
||||
self.assertEqual("CAST(cola AS TIMESTAMP)", col.sql())
|
||||
col_with_format = SF.to_timestamp("cola", "yyyy-MM-dd")
|
||||
self.assertEqual("TO_TIMESTAMP(cola, 'yyyy-MM-dd')", col_with_format.sql())
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
from unittest import mock
|
||||
|
||||
from sqlglot import ErrorLevel, ParseError, UnsupportedError, transpile
|
||||
from sqlglot import ErrorLevel, ParseError, TokenError, UnsupportedError, transpile
|
||||
from tests.dialects.test_dialect import Validator
|
||||
|
||||
|
||||
|
|
@ -8,7 +8,7 @@ class TestBigQuery(Validator):
|
|||
dialect = "bigquery"
|
||||
|
||||
def test_bigquery(self):
|
||||
with self.assertRaises(ValueError):
|
||||
with self.assertRaises(TokenError):
|
||||
transpile("'\\'", read="bigquery")
|
||||
|
||||
# Reference: https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#set_operators
|
||||
|
|
@ -520,6 +520,12 @@ class TestBigQuery(Validator):
|
|||
"bigquery": "SELECT cola, colb FROM UNNEST([STRUCT(1 AS _c0, 'test' AS _c1)])",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM UNNEST([STRUCT(1 AS id)]) CROSS JOIN UNNEST([STRUCT(1 AS id)])",
|
||||
read={
|
||||
"postgres": "SELECT * FROM (VALUES (1)) AS t1(id) CROSS JOIN (VALUES (1)) AS t2(id)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT cola, colb, colc FROM (VALUES (1, 'test', NULL)) AS tab(cola, colb, colc)",
|
||||
write={
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ class TestClickhouse(Validator):
|
|||
dialect = "clickhouse"
|
||||
|
||||
def test_clickhouse(self):
|
||||
self.validate_identity("SELECT xor(TRUE, FALSE)")
|
||||
self.validate_identity("ATTACH DATABASE DEFAULT ENGINE = ORDINARY")
|
||||
self.validate_identity("CAST(['hello'], 'Array(Enum8(''hello'' = 1))')")
|
||||
self.validate_identity("SELECT x, COUNT() FROM y GROUP BY x WITH TOTALS")
|
||||
|
|
@ -52,6 +53,34 @@ class TestClickhouse(Validator):
|
|||
"CREATE MATERIALIZED VIEW test_view (id UInt8) TO db.table1 AS SELECT * FROM test_data"
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"SELECT xor(1, 0)",
|
||||
read={
|
||||
"clickhouse": "SELECT xor(1, 0)",
|
||||
"mysql": "SELECT 1 XOR 0",
|
||||
},
|
||||
write={
|
||||
"mysql": "SELECT 1 XOR 0",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT xor(0, 1, xor(1, 0, 0))",
|
||||
write={
|
||||
"clickhouse": "SELECT xor(0, 1, xor(1, 0, 0))",
|
||||
"mysql": "SELECT 0 XOR 1 XOR 1 XOR 0 XOR 0",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT xor(xor(1, 0), 1)",
|
||||
read={
|
||||
"clickhouse": "SELECT xor(xor(1, 0), 1)",
|
||||
"mysql": "SELECT 1 XOR 0 XOR 1",
|
||||
},
|
||||
write={
|
||||
"clickhouse": "SELECT xor(xor(1, 0), 1)",
|
||||
"mysql": "SELECT 1 XOR 0 XOR 1",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"CONCAT(CASE WHEN COALESCE(CAST(a AS TEXT), '') IS NULL THEN COALESCE(CAST(a AS TEXT), '') ELSE CAST(COALESCE(CAST(a AS TEXT), '') AS TEXT) END, CASE WHEN COALESCE(CAST(b AS TEXT), '') IS NULL THEN COALESCE(CAST(b AS TEXT), '') ELSE CAST(COALESCE(CAST(b AS TEXT), '') AS TEXT) END)",
|
||||
read={"postgres": "CONCAT(a, b)"},
|
||||
|
|
@ -137,6 +166,11 @@ class TestClickhouse(Validator):
|
|||
+ " GROUP BY loyalty ORDER BY loyalty"
|
||||
},
|
||||
)
|
||||
self.validate_identity("SELECT s, arr FROM arrays_test ARRAY JOIN arr")
|
||||
self.validate_identity("SELECT s, arr, a FROM arrays_test LEFT ARRAY JOIN arr AS a")
|
||||
self.validate_identity(
|
||||
"SELECT s, arr_external FROM arrays_test ARRAY JOIN [1, 2, 3] AS arr_external"
|
||||
)
|
||||
|
||||
def test_cte(self):
|
||||
self.validate_identity("WITH 'x' AS foo SELECT foo")
|
||||
|
|
|
|||
|
|
@ -81,7 +81,7 @@ class TestDialect(Validator):
|
|||
"clickhouse": "CAST(a AS TEXT)",
|
||||
"drill": "CAST(a AS VARCHAR)",
|
||||
"duckdb": "CAST(a AS TEXT)",
|
||||
"mysql": "CAST(a AS TEXT)",
|
||||
"mysql": "CAST(a AS CHAR)",
|
||||
"hive": "CAST(a AS STRING)",
|
||||
"oracle": "CAST(a AS CLOB)",
|
||||
"postgres": "CAST(a AS TEXT)",
|
||||
|
|
@ -160,7 +160,7 @@ class TestDialect(Validator):
|
|||
"bigquery": "CAST(a AS STRING)",
|
||||
"drill": "CAST(a AS VARCHAR)",
|
||||
"duckdb": "CAST(a AS TEXT)",
|
||||
"mysql": "CAST(a AS TEXT)",
|
||||
"mysql": "CAST(a AS CHAR)",
|
||||
"hive": "CAST(a AS STRING)",
|
||||
"oracle": "CAST(a AS CLOB)",
|
||||
"postgres": "CAST(a AS TEXT)",
|
||||
|
|
@ -177,7 +177,7 @@ class TestDialect(Validator):
|
|||
"bigquery": "CAST(a AS STRING)",
|
||||
"drill": "CAST(a AS VARCHAR)",
|
||||
"duckdb": "CAST(a AS TEXT)",
|
||||
"mysql": "CAST(a AS VARCHAR)",
|
||||
"mysql": "CAST(a AS CHAR)",
|
||||
"hive": "CAST(a AS STRING)",
|
||||
"oracle": "CAST(a AS VARCHAR2)",
|
||||
"postgres": "CAST(a AS VARCHAR)",
|
||||
|
|
@ -194,7 +194,7 @@ class TestDialect(Validator):
|
|||
"bigquery": "CAST(a AS STRING)",
|
||||
"drill": "CAST(a AS VARCHAR(3))",
|
||||
"duckdb": "CAST(a AS TEXT(3))",
|
||||
"mysql": "CAST(a AS VARCHAR(3))",
|
||||
"mysql": "CAST(a AS CHAR(3))",
|
||||
"hive": "CAST(a AS VARCHAR(3))",
|
||||
"oracle": "CAST(a AS VARCHAR2(3))",
|
||||
"postgres": "CAST(a AS VARCHAR(3))",
|
||||
|
|
@ -224,13 +224,13 @@ class TestDialect(Validator):
|
|||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"TRY_CAST(a AS DOUBLE)",
|
||||
"CAST(a AS DOUBLE)",
|
||||
read={
|
||||
"postgres": "CAST(a AS DOUBLE PRECISION)",
|
||||
"redshift": "CAST(a AS DOUBLE PRECISION)",
|
||||
},
|
||||
write={
|
||||
"duckdb": "TRY_CAST(a AS DOUBLE)",
|
||||
"duckdb": "CAST(a AS DOUBLE)",
|
||||
"drill": "CAST(a AS DOUBLE)",
|
||||
"postgres": "CAST(a AS DOUBLE PRECISION)",
|
||||
"redshift": "CAST(a AS DOUBLE PRECISION)",
|
||||
|
|
@ -634,7 +634,7 @@ class TestDialect(Validator):
|
|||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"TIMESTAMP_TRUNC(TRY_CAST(x AS DATE), day)",
|
||||
"TIMESTAMP_TRUNC(CAST(x AS DATE), day)",
|
||||
read={"postgres": "DATE_TRUNC('day', x::DATE)"},
|
||||
)
|
||||
self.validate_all(
|
||||
|
|
@ -1512,6 +1512,7 @@ SELECT
|
|||
"redshift": "BEGIN",
|
||||
"snowflake": "BEGIN",
|
||||
"sqlite": "BEGIN TRANSACTION",
|
||||
"tsql": "BEGIN TRANSACTION",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
|
|
|
|||
|
|
@ -5,156 +5,9 @@ from tests.dialects.test_dialect import Validator
|
|||
class TestDuckDB(Validator):
|
||||
dialect = "duckdb"
|
||||
|
||||
def test_time(self):
|
||||
self.validate_identity("SELECT CURRENT_DATE")
|
||||
self.validate_identity("SELECT CURRENT_TIMESTAMP")
|
||||
|
||||
self.validate_all(
|
||||
"SELECT MAKE_DATE(2016, 12, 25)", read={"bigquery": "SELECT DATE(2016, 12, 25)"}
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CAST(CAST('2016-12-25 23:59:59' AS DATETIME) AS DATE)",
|
||||
read={"bigquery": "SELECT DATE(DATETIME '2016-12-25 23:59:59')"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT STRPTIME(STRFTIME(CAST(CAST('2016-12-25' AS TIMESTAMPTZ) AS DATE), '%d/%m/%Y') || ' ' || 'America/Los_Angeles', '%d/%m/%Y %Z')",
|
||||
read={
|
||||
"bigquery": "SELECT DATE(TIMESTAMP '2016-12-25', 'America/Los_Angeles')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CAST(CAST(STRPTIME('05/06/2020', '%m/%d/%Y') AS DATE) AS DATE)",
|
||||
read={"bigquery": "SELECT DATE(PARSE_DATE('%m/%d/%Y', '05/06/2020'))"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CAST('2020-01-01' AS DATE) + INTERVAL (-1) DAY",
|
||||
read={"mysql": "SELECT DATE '2020-01-01' + INTERVAL -1 DAY"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT INTERVAL '1 quarter'",
|
||||
write={"duckdb": "SELECT (90 * INTERVAL '1' day)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT ((DATE_TRUNC('DAY', CAST(CAST(DATE_TRUNC('DAY', CURRENT_TIMESTAMP) AS DATE) AS TIMESTAMP) + INTERVAL (0 - MOD((DAYOFWEEK(CAST(CAST(DATE_TRUNC('DAY', CURRENT_TIMESTAMP) AS DATE) AS TIMESTAMP)) % 7) - 1 + 7, 7)) day) + (7 * INTERVAL (-5) day))) AS t1",
|
||||
read={
|
||||
"presto": "SELECT ((DATE_ADD('week', -5, DATE_TRUNC('DAY', DATE_ADD('day', (0 - MOD((DAY_OF_WEEK(CAST(CAST(DATE_TRUNC('DAY', NOW()) AS DATE) AS TIMESTAMP)) % 7) - 1 + 7, 7)), CAST(CAST(DATE_TRUNC('DAY', NOW()) AS DATE) AS TIMESTAMP)))))) AS t1",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"EPOCH(x)",
|
||||
read={
|
||||
"presto": "TO_UNIXTIME(x)",
|
||||
},
|
||||
write={
|
||||
"bigquery": "TIME_TO_UNIX(x)",
|
||||
"duckdb": "EPOCH(x)",
|
||||
"presto": "TO_UNIXTIME(x)",
|
||||
"spark": "UNIX_TIMESTAMP(x)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"EPOCH_MS(x)",
|
||||
write={
|
||||
"bigquery": "UNIX_TO_TIME(x / 1000)",
|
||||
"duckdb": "TO_TIMESTAMP(x / 1000)",
|
||||
"presto": "FROM_UNIXTIME(x / 1000)",
|
||||
"spark": "CAST(FROM_UNIXTIME(x / 1000) AS TIMESTAMP)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"STRFTIME(x, '%y-%-m-%S')",
|
||||
write={
|
||||
"bigquery": "TIME_TO_STR(x, '%y-%-m-%S')",
|
||||
"duckdb": "STRFTIME(x, '%y-%-m-%S')",
|
||||
"postgres": "TO_CHAR(x, 'YY-FMMM-SS')",
|
||||
"presto": "DATE_FORMAT(x, '%y-%c-%S')",
|
||||
"spark": "DATE_FORMAT(x, 'yy-M-ss')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"STRFTIME(x, '%Y-%m-%d %H:%M:%S')",
|
||||
write={
|
||||
"duckdb": "STRFTIME(x, '%Y-%m-%d %H:%M:%S')",
|
||||
"presto": "DATE_FORMAT(x, '%Y-%m-%d %T')",
|
||||
"hive": "DATE_FORMAT(x, 'yyyy-MM-dd HH:mm:ss')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"STRPTIME(x, '%y-%-m')",
|
||||
write={
|
||||
"bigquery": "PARSE_TIMESTAMP('%y-%-m', x)",
|
||||
"duckdb": "STRPTIME(x, '%y-%-m')",
|
||||
"presto": "DATE_PARSE(x, '%y-%c')",
|
||||
"hive": "CAST(FROM_UNIXTIME(UNIX_TIMESTAMP(x, 'yy-M')) AS TIMESTAMP)",
|
||||
"spark": "TO_TIMESTAMP(x, 'yy-M')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"TO_TIMESTAMP(x)",
|
||||
write={
|
||||
"bigquery": "UNIX_TO_TIME(x)",
|
||||
"duckdb": "TO_TIMESTAMP(x)",
|
||||
"presto": "FROM_UNIXTIME(x)",
|
||||
"hive": "FROM_UNIXTIME(x)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"STRPTIME(x, '%-m/%-d/%y %-I:%M %p')",
|
||||
write={
|
||||
"bigquery": "PARSE_TIMESTAMP('%-m/%-d/%y %-I:%M %p', x)",
|
||||
"duckdb": "STRPTIME(x, '%-m/%-d/%y %-I:%M %p')",
|
||||
"presto": "DATE_PARSE(x, '%c/%e/%y %l:%i %p')",
|
||||
"hive": "CAST(FROM_UNIXTIME(UNIX_TIMESTAMP(x, 'M/d/yy h:mm a')) AS TIMESTAMP)",
|
||||
"spark": "TO_TIMESTAMP(x, 'M/d/yy h:mm a')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"CAST(start AS TIMESTAMPTZ) AT TIME ZONE 'America/New_York'",
|
||||
read={
|
||||
"snowflake": "CONVERT_TIMEZONE('America/New_York', CAST(start AS TIMESTAMPTZ))",
|
||||
},
|
||||
write={
|
||||
"bigquery": "TIMESTAMP(DATETIME(CAST(start AS TIMESTAMP), 'America/New_York'))",
|
||||
"duckdb": "CAST(start AS TIMESTAMPTZ) AT TIME ZONE 'America/New_York'",
|
||||
"snowflake": "CONVERT_TIMEZONE('America/New_York', CAST(start AS TIMESTAMPTZ))",
|
||||
},
|
||||
)
|
||||
|
||||
def test_sample(self):
|
||||
self.validate_all(
|
||||
"SELECT * FROM tbl USING SAMPLE 5",
|
||||
write={"duckdb": "SELECT * FROM tbl USING SAMPLE (5)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM tbl USING SAMPLE 10%",
|
||||
write={"duckdb": "SELECT * FROM tbl USING SAMPLE (10 PERCENT)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM tbl USING SAMPLE 10 PERCENT (bernoulli)",
|
||||
write={"duckdb": "SELECT * FROM tbl USING SAMPLE BERNOULLI (10 PERCENT)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM tbl USING SAMPLE reservoir(50 ROWS) REPEATABLE (100)",
|
||||
write={"duckdb": "SELECT * FROM tbl USING SAMPLE RESERVOIR (50 ROWS) REPEATABLE (100)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM tbl USING SAMPLE 10% (system, 377)",
|
||||
write={"duckdb": "SELECT * FROM tbl USING SAMPLE SYSTEM (10 PERCENT) REPEATABLE (377)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM tbl TABLESAMPLE RESERVOIR(20%), tbl2 WHERE tbl.i=tbl2.i",
|
||||
write={
|
||||
"duckdb": "SELECT * FROM tbl TABLESAMPLE RESERVOIR (20 PERCENT), tbl2 WHERE tbl.i = tbl2.i"
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM tbl, tbl2 WHERE tbl.i=tbl2.i USING SAMPLE RESERVOIR(20%)",
|
||||
write={
|
||||
"duckdb": "SELECT * FROM tbl, tbl2 WHERE tbl.i = tbl2.i USING SAMPLE RESERVOIR (20 PERCENT)"
|
||||
},
|
||||
)
|
||||
|
||||
def test_duckdb(self):
|
||||
self.validate_identity("SELECT SUM(x) FILTER (x = 1)", "SELECT SUM(x) FILTER(WHERE x = 1)")
|
||||
|
||||
# https://github.com/duckdb/duckdb/releases/tag/v0.8.0
|
||||
self.assertEqual(
|
||||
parse_one("a / b", read="duckdb").assert_is(exp.Div).sql(dialect="duckdb"), "a / b"
|
||||
|
|
@ -203,6 +56,31 @@ class TestDuckDB(Validator):
|
|||
self.validate_all("0x1010", write={"": "0 AS x1010"})
|
||||
self.validate_all("x ~ y", write={"duckdb": "REGEXP_MATCHES(x, y)"})
|
||||
self.validate_all("SELECT * FROM 'x.y'", write={"duckdb": 'SELECT * FROM "x.y"'})
|
||||
self.validate_all(
|
||||
"DATE_DIFF('day', CAST(b AS DATE), CAST(a AS DATE))",
|
||||
read={
|
||||
"duckdb": "DATE_DIFF('day', CAST(b AS DATE), CAST(a AS DATE))",
|
||||
"hive": "DATEDIFF(a, b)",
|
||||
"spark": "DATEDIFF(a, b)",
|
||||
"spark2": "DATEDIFF(a, b)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"XOR(a, b)",
|
||||
read={
|
||||
"": "a ^ b",
|
||||
"bigquery": "a ^ b",
|
||||
"presto": "BITWISE_XOR(a, b)",
|
||||
"postgres": "a # b",
|
||||
},
|
||||
write={
|
||||
"": "a ^ b",
|
||||
"bigquery": "a ^ b",
|
||||
"duckdb": "XOR(a, b)",
|
||||
"presto": "BITWISE_XOR(a, b)",
|
||||
"postgres": "a # b",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"PIVOT_WIDER Cities ON Year USING SUM(Population)",
|
||||
write={"duckdb": "PIVOT Cities ON Year USING SUM(Population)"},
|
||||
|
|
@ -459,6 +337,155 @@ class TestDuckDB(Validator):
|
|||
unsupported_level=ErrorLevel.IMMEDIATE,
|
||||
)
|
||||
|
||||
def test_time(self):
|
||||
self.validate_identity("SELECT CURRENT_DATE")
|
||||
self.validate_identity("SELECT CURRENT_TIMESTAMP")
|
||||
|
||||
self.validate_all(
|
||||
"SELECT MAKE_DATE(2016, 12, 25)", read={"bigquery": "SELECT DATE(2016, 12, 25)"}
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CAST(CAST('2016-12-25 23:59:59' AS DATETIME) AS DATE)",
|
||||
read={"bigquery": "SELECT DATE(DATETIME '2016-12-25 23:59:59')"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT STRPTIME(STRFTIME(CAST(CAST('2016-12-25' AS TIMESTAMPTZ) AS DATE), '%d/%m/%Y') || ' ' || 'America/Los_Angeles', '%d/%m/%Y %Z')",
|
||||
read={
|
||||
"bigquery": "SELECT DATE(TIMESTAMP '2016-12-25', 'America/Los_Angeles')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CAST(CAST(STRPTIME('05/06/2020', '%m/%d/%Y') AS DATE) AS DATE)",
|
||||
read={"bigquery": "SELECT DATE(PARSE_DATE('%m/%d/%Y', '05/06/2020'))"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CAST('2020-01-01' AS DATE) + INTERVAL (-1) DAY",
|
||||
read={"mysql": "SELECT DATE '2020-01-01' + INTERVAL -1 DAY"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT INTERVAL '1 quarter'",
|
||||
write={"duckdb": "SELECT (90 * INTERVAL '1' day)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT ((DATE_TRUNC('DAY', CAST(CAST(DATE_TRUNC('DAY', CURRENT_TIMESTAMP) AS DATE) AS TIMESTAMP) + INTERVAL (0 - MOD((DAYOFWEEK(CAST(CAST(DATE_TRUNC('DAY', CURRENT_TIMESTAMP) AS DATE) AS TIMESTAMP)) % 7) - 1 + 7, 7)) day) + (7 * INTERVAL (-5) day))) AS t1",
|
||||
read={
|
||||
"presto": "SELECT ((DATE_ADD('week', -5, DATE_TRUNC('DAY', DATE_ADD('day', (0 - MOD((DAY_OF_WEEK(CAST(CAST(DATE_TRUNC('DAY', NOW()) AS DATE) AS TIMESTAMP)) % 7) - 1 + 7, 7)), CAST(CAST(DATE_TRUNC('DAY', NOW()) AS DATE) AS TIMESTAMP)))))) AS t1",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"EPOCH(x)",
|
||||
read={
|
||||
"presto": "TO_UNIXTIME(x)",
|
||||
},
|
||||
write={
|
||||
"bigquery": "TIME_TO_UNIX(x)",
|
||||
"duckdb": "EPOCH(x)",
|
||||
"presto": "TO_UNIXTIME(x)",
|
||||
"spark": "UNIX_TIMESTAMP(x)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"EPOCH_MS(x)",
|
||||
write={
|
||||
"bigquery": "UNIX_TO_TIME(x / 1000)",
|
||||
"duckdb": "TO_TIMESTAMP(x / 1000)",
|
||||
"presto": "FROM_UNIXTIME(x / 1000)",
|
||||
"spark": "CAST(FROM_UNIXTIME(x / 1000) AS TIMESTAMP)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"STRFTIME(x, '%y-%-m-%S')",
|
||||
write={
|
||||
"bigquery": "TIME_TO_STR(x, '%y-%-m-%S')",
|
||||
"duckdb": "STRFTIME(x, '%y-%-m-%S')",
|
||||
"postgres": "TO_CHAR(x, 'YY-FMMM-SS')",
|
||||
"presto": "DATE_FORMAT(x, '%y-%c-%S')",
|
||||
"spark": "DATE_FORMAT(x, 'yy-M-ss')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"STRFTIME(x, '%Y-%m-%d %H:%M:%S')",
|
||||
write={
|
||||
"duckdb": "STRFTIME(x, '%Y-%m-%d %H:%M:%S')",
|
||||
"presto": "DATE_FORMAT(x, '%Y-%m-%d %T')",
|
||||
"hive": "DATE_FORMAT(x, 'yyyy-MM-dd HH:mm:ss')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"STRPTIME(x, '%y-%-m')",
|
||||
write={
|
||||
"bigquery": "PARSE_TIMESTAMP('%y-%-m', x)",
|
||||
"duckdb": "STRPTIME(x, '%y-%-m')",
|
||||
"presto": "DATE_PARSE(x, '%y-%c')",
|
||||
"hive": "CAST(FROM_UNIXTIME(UNIX_TIMESTAMP(x, 'yy-M')) AS TIMESTAMP)",
|
||||
"spark": "TO_TIMESTAMP(x, 'yy-M')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"TO_TIMESTAMP(x)",
|
||||
write={
|
||||
"bigquery": "UNIX_TO_TIME(x)",
|
||||
"duckdb": "TO_TIMESTAMP(x)",
|
||||
"presto": "FROM_UNIXTIME(x)",
|
||||
"hive": "FROM_UNIXTIME(x)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"STRPTIME(x, '%-m/%-d/%y %-I:%M %p')",
|
||||
write={
|
||||
"bigquery": "PARSE_TIMESTAMP('%-m/%-d/%y %-I:%M %p', x)",
|
||||
"duckdb": "STRPTIME(x, '%-m/%-d/%y %-I:%M %p')",
|
||||
"presto": "DATE_PARSE(x, '%c/%e/%y %l:%i %p')",
|
||||
"hive": "CAST(FROM_UNIXTIME(UNIX_TIMESTAMP(x, 'M/d/yy h:mm a')) AS TIMESTAMP)",
|
||||
"spark": "TO_TIMESTAMP(x, 'M/d/yy h:mm a')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"CAST(start AS TIMESTAMPTZ) AT TIME ZONE 'America/New_York'",
|
||||
read={
|
||||
"snowflake": "CONVERT_TIMEZONE('America/New_York', CAST(start AS TIMESTAMPTZ))",
|
||||
},
|
||||
write={
|
||||
"bigquery": "TIMESTAMP(DATETIME(CAST(start AS TIMESTAMP), 'America/New_York'))",
|
||||
"duckdb": "CAST(start AS TIMESTAMPTZ) AT TIME ZONE 'America/New_York'",
|
||||
"snowflake": "CONVERT_TIMEZONE('America/New_York', CAST(start AS TIMESTAMPTZ))",
|
||||
},
|
||||
)
|
||||
|
||||
def test_sample(self):
|
||||
self.validate_all(
|
||||
"SELECT * FROM tbl USING SAMPLE 5",
|
||||
write={"duckdb": "SELECT * FROM tbl USING SAMPLE (5)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM tbl USING SAMPLE 10%",
|
||||
write={"duckdb": "SELECT * FROM tbl USING SAMPLE (10 PERCENT)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM tbl USING SAMPLE 10 PERCENT (bernoulli)",
|
||||
write={"duckdb": "SELECT * FROM tbl USING SAMPLE BERNOULLI (10 PERCENT)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM tbl USING SAMPLE reservoir(50 ROWS) REPEATABLE (100)",
|
||||
write={"duckdb": "SELECT * FROM tbl USING SAMPLE RESERVOIR (50 ROWS) REPEATABLE (100)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM tbl USING SAMPLE 10% (system, 377)",
|
||||
write={"duckdb": "SELECT * FROM tbl USING SAMPLE SYSTEM (10 PERCENT) REPEATABLE (377)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM tbl TABLESAMPLE RESERVOIR(20%), tbl2 WHERE tbl.i=tbl2.i",
|
||||
write={
|
||||
"duckdb": "SELECT * FROM tbl TABLESAMPLE RESERVOIR (20 PERCENT), tbl2 WHERE tbl.i = tbl2.i"
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM tbl, tbl2 WHERE tbl.i=tbl2.i USING SAMPLE RESERVOIR(20%)",
|
||||
write={
|
||||
"duckdb": "SELECT * FROM tbl, tbl2 WHERE tbl.i = tbl2.i USING SAMPLE RESERVOIR (20 PERCENT)"
|
||||
},
|
||||
)
|
||||
|
||||
def test_array(self):
|
||||
self.validate_identity("ARRAY(SELECT id FROM t)")
|
||||
self.validate_identity("ARRAY((SELECT id FROM t))")
|
||||
|
|
|
|||
|
|
@ -7,28 +7,57 @@ class TestHive(Validator):
|
|||
def test_bits(self):
|
||||
self.validate_all(
|
||||
"x & 1",
|
||||
write={
|
||||
read={
|
||||
"duckdb": "x & 1",
|
||||
"presto": "BITWISE_AND(x, 1)",
|
||||
"spark": "x & 1",
|
||||
},
|
||||
write={
|
||||
"duckdb": "x & 1",
|
||||
"hive": "x & 1",
|
||||
"presto": "BITWISE_AND(x, 1)",
|
||||
"spark": "x & 1",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"~x",
|
||||
"x & 1 > 0",
|
||||
read={
|
||||
"duckdb": "x & 1 > 0",
|
||||
"presto": "BITWISE_AND(x, 1) > 0",
|
||||
"spark": "x & 1 > 0",
|
||||
},
|
||||
write={
|
||||
"duckdb": "x & 1 > 0",
|
||||
"presto": "BITWISE_AND(x, 1) > 0",
|
||||
"hive": "x & 1 > 0",
|
||||
"spark": "x & 1 > 0",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"~x",
|
||||
read={
|
||||
"duckdb": "~x",
|
||||
"presto": "BITWISE_NOT(x)",
|
||||
"spark": "~x",
|
||||
},
|
||||
write={
|
||||
"duckdb": "~x",
|
||||
"hive": "~x",
|
||||
"presto": "BITWISE_NOT(x)",
|
||||
"spark": "~x",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"x | 1",
|
||||
write={
|
||||
read={
|
||||
"duckdb": "x | 1",
|
||||
"presto": "BITWISE_OR(x, 1)",
|
||||
"spark": "x | 1",
|
||||
},
|
||||
write={
|
||||
"duckdb": "x | 1",
|
||||
"hive": "x | 1",
|
||||
"presto": "BITWISE_OR(x, 1)",
|
||||
"spark": "x | 1",
|
||||
},
|
||||
)
|
||||
|
|
@ -56,15 +85,6 @@ class TestHive(Validator):
|
|||
"spark": "SHIFTRIGHT(x, 1)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"x & 1 > 0",
|
||||
write={
|
||||
"duckdb": "x & 1 > 0",
|
||||
"presto": "BITWISE_AND(x, 1) > 0",
|
||||
"hive": "x & 1 > 0",
|
||||
"spark": "x & 1 > 0",
|
||||
},
|
||||
)
|
||||
|
||||
def test_cast(self):
|
||||
self.validate_all(
|
||||
|
|
|
|||
|
|
@ -83,7 +83,7 @@ class TestMySQL(Validator):
|
|||
self.validate_identity("CAST(x AS SET('a', 'b'))")
|
||||
self.validate_identity("SELECT CURRENT_TIMESTAMP(6)")
|
||||
self.validate_identity("x ->> '$.name'")
|
||||
self.validate_identity("SELECT CAST(`a`.`b` AS INT) FROM foo")
|
||||
self.validate_identity("SELECT CAST(`a`.`b` AS CHAR) FROM foo")
|
||||
self.validate_identity("SELECT TRIM(LEADING 'bla' FROM ' XXX ')")
|
||||
self.validate_identity("SELECT TRIM(TRAILING 'bla' FROM ' XXX ')")
|
||||
self.validate_identity("SELECT TRIM(BOTH 'bla' FROM ' XXX ')")
|
||||
|
|
@ -436,6 +436,14 @@ class TestMySQL(Validator):
|
|||
self.validate_identity("TIME_STR_TO_UNIX(x)", "UNIX_TIMESTAMP(x)")
|
||||
|
||||
def test_mysql(self):
|
||||
self.validate_all(
|
||||
"CAST(x AS TEXT)",
|
||||
write={
|
||||
"mysql": "CAST(x AS CHAR)",
|
||||
"presto": "CAST(x AS VARCHAR)",
|
||||
"starrocks": "CAST(x AS STRING)",
|
||||
},
|
||||
)
|
||||
self.validate_all("CAST(x AS SIGNED)", write={"mysql": "CAST(x AS SIGNED)"})
|
||||
self.validate_all("CAST(x AS SIGNED INTEGER)", write={"mysql": "CAST(x AS SIGNED)"})
|
||||
self.validate_all("CAST(x AS UNSIGNED)", write={"mysql": "CAST(x AS UNSIGNED)"})
|
||||
|
|
|
|||
|
|
@ -255,8 +255,8 @@ class TestPostgres(Validator):
|
|||
"GENERATE_SERIES('2019-01-01'::TIMESTAMP, NOW(), '1day')",
|
||||
write={
|
||||
"postgres": "GENERATE_SERIES(CAST('2019-01-01' AS TIMESTAMP), CURRENT_TIMESTAMP, INTERVAL '1 day')",
|
||||
"presto": "SEQUENCE(TRY_CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP AS TIMESTAMP), INTERVAL '1' day)",
|
||||
"trino": "SEQUENCE(TRY_CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP AS TIMESTAMP), INTERVAL '1' day)",
|
||||
"presto": "SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP AS TIMESTAMP), INTERVAL '1' day)",
|
||||
"trino": "SEQUENCE(CAST('2019-01-01' AS TIMESTAMP), CAST(CURRENT_TIMESTAMP AS TIMESTAMP), INTERVAL '1' day)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
|
|
@ -470,7 +470,7 @@ class TestPostgres(Validator):
|
|||
"""SELECT JSON_ARRAY_ELEMENTS((foo->'sections')::JSON) AS sections""",
|
||||
write={
|
||||
"postgres": """SELECT JSON_ARRAY_ELEMENTS(CAST((foo -> 'sections') AS JSON)) AS sections""",
|
||||
"presto": """SELECT JSON_ARRAY_ELEMENTS(TRY_CAST((JSON_EXTRACT(foo, 'sections')) AS JSON)) AS sections""",
|
||||
"presto": """SELECT JSON_ARRAY_ELEMENTS(CAST((JSON_EXTRACT(foo, 'sections')) AS JSON)) AS sections""",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
|
|
@ -570,7 +570,7 @@ class TestPostgres(Validator):
|
|||
},
|
||||
)
|
||||
|
||||
self.assertIsInstance(parse_one("id::UUID", read="postgres"), exp.TryCast)
|
||||
self.assertIsInstance(parse_one("id::UUID", read="postgres"), exp.Cast)
|
||||
|
||||
def test_bool_or(self):
|
||||
self.validate_all(
|
||||
|
|
|
|||
|
|
@ -8,6 +8,14 @@ class TestPresto(Validator):
|
|||
dialect = "presto"
|
||||
|
||||
def test_cast(self):
|
||||
self.validate_all(
|
||||
"SELECT CAST('10C' AS INTEGER)",
|
||||
read={
|
||||
"postgres": "SELECT CAST('10C' AS INTEGER)",
|
||||
"presto": "SELECT CAST('10C' AS INTEGER)",
|
||||
"redshift": "SELECT CAST('10C' AS INTEGER)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT DATE_DIFF('week', CAST(CAST('2009-01-01' AS TIMESTAMP) AS DATE), CAST(CAST('2009-12-31' AS TIMESTAMP) AS DATE))",
|
||||
read={"redshift": "SELECT DATEDIFF(week, '2009-01-01', '2009-12-31')"},
|
||||
|
|
@ -17,7 +25,7 @@ class TestPresto(Validator):
|
|||
read={"redshift": "SELECT DATEADD(month, 18, '2008-02-28')"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT TRY_CAST('1970-01-01 00:00:00' AS TIMESTAMP)",
|
||||
"SELECT CAST('1970-01-01 00:00:00' AS TIMESTAMP)",
|
||||
read={"postgres": "SELECT 'epoch'::TIMESTAMP"},
|
||||
)
|
||||
self.validate_all(
|
||||
|
|
|
|||
|
|
@ -10,6 +10,13 @@ class TestRedshift(Validator):
|
|||
self.validate_identity("foo$")
|
||||
self.validate_identity("$foo")
|
||||
|
||||
self.validate_all(
|
||||
"SELECT ADD_MONTHS('2008-03-31', 1)",
|
||||
write={
|
||||
"redshift": "SELECT DATEADD(month, 1, '2008-03-31')",
|
||||
"trino": "SELECT DATE_ADD('month', 1, CAST(CAST('2008-03-31' AS TIMESTAMP) AS DATE))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT STRTOL('abc', 16)",
|
||||
read={
|
||||
|
|
@ -104,7 +111,7 @@ class TestRedshift(Validator):
|
|||
"SELECT ST_AsEWKT(ST_GeomFromEWKT('SRID=4326;POINT(10 20)')::geography)",
|
||||
write={
|
||||
"redshift": "SELECT ST_ASEWKT(CAST(ST_GEOMFROMEWKT('SRID=4326;POINT(10 20)') AS GEOGRAPHY))",
|
||||
"bigquery": "SELECT ST_AsEWKT(SAFE_CAST(ST_GeomFromEWKT('SRID=4326;POINT(10 20)') AS GEOGRAPHY))",
|
||||
"bigquery": "SELECT ST_AsEWKT(CAST(ST_GeomFromEWKT('SRID=4326;POINT(10 20)') AS GEOGRAPHY))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
|
|
@ -222,7 +229,11 @@ class TestRedshift(Validator):
|
|||
def test_values(self):
|
||||
self.validate_all(
|
||||
"SELECT * FROM (VALUES (1, 2)) AS t",
|
||||
write={"redshift": "SELECT * FROM (SELECT 1, 2) AS t"},
|
||||
write={
|
||||
"redshift": "SELECT * FROM (SELECT 1, 2) AS t",
|
||||
"mysql": "SELECT * FROM (SELECT 1, 2) AS t",
|
||||
"presto": "SELECT * FROM (VALUES (1, 2)) AS t",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT * FROM (VALUES (1)) AS t1(id) CROSS JOIN (VALUES (1)) AS t2(id)",
|
||||
|
|
|
|||
|
|
@ -386,7 +386,9 @@ class TestSnowflake(Validator):
|
|||
self.validate_all(
|
||||
"SELECT RLIKE(a, b)",
|
||||
write={
|
||||
"hive": "SELECT a RLIKE b",
|
||||
"snowflake": "SELECT REGEXP_LIKE(a, b)",
|
||||
"spark": "SELECT a RLIKE b",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
|
|
@ -913,6 +915,58 @@ FROM persons AS p, LATERAL FLATTEN(input => p.c, path => 'contact') AS f, LATERA
|
|||
},
|
||||
)
|
||||
|
||||
@mock.patch("sqlglot.generator.logger")
|
||||
def test_regexp_replace(self, logger):
|
||||
self.validate_all(
|
||||
"REGEXP_REPLACE(subject, pattern)",
|
||||
write={
|
||||
"bigquery": "REGEXP_REPLACE(subject, pattern, '')",
|
||||
"duckdb": "REGEXP_REPLACE(subject, pattern, '')",
|
||||
"hive": "REGEXP_REPLACE(subject, pattern, '')",
|
||||
"snowflake": "REGEXP_REPLACE(subject, pattern, '')",
|
||||
"spark": "REGEXP_REPLACE(subject, pattern, '')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
read={
|
||||
"bigquery": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
"duckdb": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
"hive": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
"spark": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
},
|
||||
write={
|
||||
"bigquery": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
"duckdb": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
"hive": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
"snowflake": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
"spark": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"REGEXP_REPLACE(subject, pattern, replacement, position)",
|
||||
read={
|
||||
"spark": "REGEXP_REPLACE(subject, pattern, replacement, position)",
|
||||
},
|
||||
write={
|
||||
"bigquery": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
"duckdb": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
"hive": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
"snowflake": "REGEXP_REPLACE(subject, pattern, replacement, position)",
|
||||
"spark": "REGEXP_REPLACE(subject, pattern, replacement, position)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"REGEXP_REPLACE(subject, pattern, replacement, position, occurrence, parameters)",
|
||||
write={
|
||||
"bigquery": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
"duckdb": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
"hive": "REGEXP_REPLACE(subject, pattern, replacement)",
|
||||
"snowflake": "REGEXP_REPLACE(subject, pattern, replacement, position, occurrence, parameters)",
|
||||
"spark": "REGEXP_REPLACE(subject, pattern, replacement, position)",
|
||||
},
|
||||
)
|
||||
|
||||
def test_match_recognize(self):
|
||||
for row in (
|
||||
"ONE ROW PER MATCH",
|
||||
|
|
|
|||
|
|
@ -224,6 +224,9 @@ TBLPROPERTIES (
|
|||
)
|
||||
|
||||
def test_spark(self):
|
||||
self.validate_identity("SELECT TRANSFORM(ARRAY(1, 2, 3), x -> x + 1)")
|
||||
self.validate_identity("SELECT TRANSFORM(ARRAY(1, 2, 3), (x, i) -> x + i)")
|
||||
self.validate_identity("REFRESH table a.b.c")
|
||||
self.validate_identity("INTERVAL -86 days")
|
||||
self.validate_identity("SELECT UNIX_TIMESTAMP()")
|
||||
self.validate_identity("TRIM(' SparkSQL ')")
|
||||
|
|
@ -232,6 +235,61 @@ TBLPROPERTIES (
|
|||
self.validate_identity("TRIM(TRAILING 'SL' FROM 'SSparkSQLS')")
|
||||
self.validate_identity("SPLIT(str, pattern, lim)")
|
||||
|
||||
self.validate_all(
|
||||
"SELECT DATEDIFF(month, CAST('1996-10-30' AS TIMESTAMP), CAST('1997-02-28 10:30:00' AS TIMESTAMP))",
|
||||
read={
|
||||
"duckdb": "SELECT DATEDIFF('month', CAST('1996-10-30' AS TIMESTAMP), CAST('1997-02-28 10:30:00' AS TIMESTAMP))",
|
||||
},
|
||||
write={
|
||||
"spark": "SELECT DATEDIFF(month, TO_DATE(CAST('1996-10-30' AS TIMESTAMP)), TO_DATE(CAST('1997-02-28 10:30:00' AS TIMESTAMP)))",
|
||||
"spark2": "SELECT MONTHS_BETWEEN(TO_DATE(CAST('1997-02-28 10:30:00' AS TIMESTAMP)), TO_DATE(CAST('1996-10-30' AS TIMESTAMP)))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT MONTHS_BETWEEN('1997-02-28 10:30:00', '1996-10-30')",
|
||||
write={
|
||||
"duckdb": "SELECT DATEDIFF('month', CAST('1996-10-30' AS TIMESTAMP), CAST('1997-02-28 10:30:00' AS TIMESTAMP))",
|
||||
"hive": "SELECT MONTHS_BETWEEN('1997-02-28 10:30:00', '1996-10-30')",
|
||||
"spark": "SELECT MONTHS_BETWEEN('1997-02-28 10:30:00', '1996-10-30')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT MONTHS_BETWEEN('1997-02-28 10:30:00', '1996-10-30', FALSE)",
|
||||
write={
|
||||
"duckdb": "SELECT DATEDIFF('month', CAST('1996-10-30' AS TIMESTAMP), CAST('1997-02-28 10:30:00' AS TIMESTAMP))",
|
||||
"hive": "SELECT MONTHS_BETWEEN('1997-02-28 10:30:00', '1996-10-30')",
|
||||
"spark": "SELECT MONTHS_BETWEEN('1997-02-28 10:30:00', '1996-10-30', FALSE)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT TO_TIMESTAMP('2016-12-31 00:12:00')",
|
||||
write={
|
||||
"": "SELECT CAST('2016-12-31 00:12:00' AS TIMESTAMP)",
|
||||
"duckdb": "SELECT CAST('2016-12-31 00:12:00' AS TIMESTAMP)",
|
||||
"spark": "SELECT CAST('2016-12-31 00:12:00' AS TIMESTAMP)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT TO_TIMESTAMP('2016-12-31', 'yyyy-MM-dd')",
|
||||
read={
|
||||
"duckdb": "SELECT STRPTIME('2016-12-31', '%Y-%m-%d')",
|
||||
},
|
||||
write={
|
||||
"": "SELECT STR_TO_TIME('2016-12-31', '%Y-%m-%d')",
|
||||
"duckdb": "SELECT STRPTIME('2016-12-31', '%Y-%m-%d')",
|
||||
"spark": "SELECT TO_TIMESTAMP('2016-12-31', 'yyyy-MM-dd')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT RLIKE('John Doe', 'John.*')",
|
||||
write={
|
||||
"bigquery": "SELECT REGEXP_CONTAINS('John Doe', 'John.*')",
|
||||
"hive": "SELECT 'John Doe' RLIKE 'John.*'",
|
||||
"postgres": "SELECT 'John Doe' ~ 'John.*'",
|
||||
"snowflake": "SELECT REGEXP_LIKE('John Doe', 'John.*')",
|
||||
"spark": "SELECT 'John Doe' RLIKE 'John.*'",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"UNHEX(MD5(x))",
|
||||
write={
|
||||
|
|
@ -446,3 +504,21 @@ TBLPROPERTIES (
|
|||
"CURRENT_USER()",
|
||||
write={"spark": "CURRENT_USER()"},
|
||||
)
|
||||
|
||||
def test_transform_query(self):
|
||||
self.validate_identity("SELECT TRANSFORM(x) USING 'x' AS (x INT) FROM t")
|
||||
self.validate_identity(
|
||||
"SELECT TRANSFORM(zip_code, name, age) USING 'cat' AS (a, b, c) FROM person WHERE zip_code > 94511"
|
||||
)
|
||||
self.validate_identity(
|
||||
"SELECT TRANSFORM(zip_code, name, age) USING 'cat' AS (a STRING, b STRING, c STRING) FROM person WHERE zip_code > 94511"
|
||||
)
|
||||
self.validate_identity(
|
||||
"SELECT TRANSFORM(name, age) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' NULL DEFINED AS 'NULL' USING 'cat' AS (name_age STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '@' LINES TERMINATED BY '\n' NULL DEFINED AS 'NULL' FROM person"
|
||||
)
|
||||
self.validate_identity(
|
||||
"SELECT TRANSFORM(zip_code, name, age) ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' WITH SERDEPROPERTIES ('field.delim'='\t') USING 'cat' AS (a STRING, b STRING, c STRING) ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' WITH SERDEPROPERTIES ('field.delim'='\t') FROM person WHERE zip_code > 94511"
|
||||
)
|
||||
self.validate_identity(
|
||||
"SELECT TRANSFORM(zip_code, name, age) USING 'cat' FROM person WHERE zip_code > 94500"
|
||||
)
|
||||
|
|
|
|||
|
|
@ -61,6 +61,9 @@ class TestTeradata(Validator):
|
|||
self.validate_identity(
|
||||
"CREATE VOLATILE MULTISET TABLE a, NOT LOCAL AFTER JOURNAL, FREESPACE=1 PERCENT, DATABLOCKSIZE=10 BYTES, WITH NO CONCURRENT ISOLATED LOADING FOR ALL (a INT)"
|
||||
)
|
||||
self.validate_identity(
|
||||
"CREATE VOLATILE SET TABLE example1 AS (SELECT col1, col2, col3 FROM table1) WITH DATA PRIMARY INDEX (col1) ON COMMIT PRESERVE ROWS"
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"""
|
||||
|
|
|
|||
|
|
@ -33,15 +33,15 @@ class TestTSQL(Validator):
|
|||
)
|
||||
|
||||
self.validate_all(
|
||||
"SELECT DATEPART(year, TRY_CAST('2017-01-01' AS DATE))",
|
||||
"SELECT DATEPART(year, CAST('2017-01-01' AS DATE))",
|
||||
read={"postgres": "SELECT DATE_PART('year', '2017-01-01'::DATE)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT DATEPART(month, TRY_CAST('2017-03-01' AS DATE))",
|
||||
"SELECT DATEPART(month, CAST('2017-03-01' AS DATE))",
|
||||
read={"postgres": "SELECT DATE_PART('month', '2017-03-01'::DATE)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT DATEPART(day, TRY_CAST('2017-01-02' AS DATE))",
|
||||
"SELECT DATEPART(day, CAST('2017-01-02' AS DATE))",
|
||||
read={"postgres": "SELECT DATE_PART('day', '2017-01-02'::DATE)"},
|
||||
)
|
||||
self.validate_all(
|
||||
|
|
@ -54,7 +54,7 @@ class TestTSQL(Validator):
|
|||
self.validate_all(
|
||||
"CONVERT(INT, CONVERT(NUMERIC, '444.75'))",
|
||||
write={
|
||||
"mysql": "CAST(CAST('444.75' AS DECIMAL) AS INT)",
|
||||
"mysql": "CAST(CAST('444.75' AS DECIMAL) AS SIGNED)",
|
||||
"tsql": "CAST(CAST('444.75' AS NUMERIC) AS INTEGER)",
|
||||
},
|
||||
)
|
||||
|
|
@ -389,7 +389,58 @@ class TestTSQL(Validator):
|
|||
},
|
||||
)
|
||||
|
||||
def test_ddl(self):
|
||||
self.validate_all(
|
||||
"CREATE TABLE #mytemp (a INTEGER, b CHAR(2), c TIME(4), d FLOAT(24))",
|
||||
write={
|
||||
"tsql": "CREATE TABLE #mytemp (a INTEGER, b CHAR(2), c TIMESTAMP(4), d FLOAT(24))"
|
||||
},
|
||||
)
|
||||
|
||||
def test_transaction(self):
|
||||
# BEGIN { TRAN | TRANSACTION }
|
||||
# [ { transaction_name | @tran_name_variable }
|
||||
# [ WITH MARK [ 'description' ] ]
|
||||
# ]
|
||||
# [ ; ]
|
||||
self.validate_identity("BEGIN TRANSACTION")
|
||||
self.validate_all("BEGIN TRAN", write={"tsql": "BEGIN TRANSACTION"})
|
||||
self.validate_identity("BEGIN TRANSACTION transaction_name")
|
||||
self.validate_identity("BEGIN TRANSACTION @tran_name_variable")
|
||||
self.validate_identity("BEGIN TRANSACTION transaction_name WITH MARK 'description'")
|
||||
|
||||
def test_commit(self):
|
||||
# COMMIT [ { TRAN | TRANSACTION } [ transaction_name | @tran_name_variable ] ] [ WITH ( DELAYED_DURABILITY = { OFF | ON } ) ] [ ; ]
|
||||
|
||||
self.validate_all("COMMIT", write={"tsql": "COMMIT TRANSACTION"})
|
||||
self.validate_all("COMMIT TRAN", write={"tsql": "COMMIT TRANSACTION"})
|
||||
self.validate_identity("COMMIT TRANSACTION")
|
||||
self.validate_identity("COMMIT TRANSACTION transaction_name")
|
||||
self.validate_identity("COMMIT TRANSACTION @tran_name_variable")
|
||||
|
||||
self.validate_identity(
|
||||
"COMMIT TRANSACTION @tran_name_variable WITH (DELAYED_DURABILITY = ON)"
|
||||
)
|
||||
self.validate_identity(
|
||||
"COMMIT TRANSACTION transaction_name WITH (DELAYED_DURABILITY = OFF)"
|
||||
)
|
||||
|
||||
def test_rollback(self):
|
||||
# Applies to SQL Server and Azure SQL Database
|
||||
# ROLLBACK { TRAN | TRANSACTION }
|
||||
# [ transaction_name | @tran_name_variable
|
||||
# | savepoint_name | @savepoint_variable ]
|
||||
# [ ; ]
|
||||
self.validate_all("ROLLBACK", write={"tsql": "ROLLBACK TRANSACTION"})
|
||||
self.validate_all("ROLLBACK TRAN", write={"tsql": "ROLLBACK TRANSACTION"})
|
||||
self.validate_identity("ROLLBACK TRANSACTION")
|
||||
self.validate_identity("ROLLBACK TRANSACTION transaction_name")
|
||||
self.validate_identity("ROLLBACK TRANSACTION @tran_name_variable")
|
||||
|
||||
def test_udf(self):
|
||||
self.validate_identity(
|
||||
"DECLARE @DWH_DateCreated DATETIME = CONVERT(DATETIME, getdate(), 104)"
|
||||
)
|
||||
self.validate_identity(
|
||||
"CREATE PROCEDURE foo @a INTEGER, @b INTEGER AS SELECT @a = SUM(bla) FROM baz AS bar"
|
||||
)
|
||||
|
|
@ -446,6 +497,12 @@ WHERE
|
|||
pretty=True,
|
||||
)
|
||||
|
||||
def test_procedure_keywords(self):
|
||||
self.validate_identity("BEGIN")
|
||||
self.validate_identity("END")
|
||||
self.validate_identity("SET XACT_ABORT ON")
|
||||
|
||||
def test_fullproc(self):
|
||||
sql = """
|
||||
CREATE procedure [TRANSF].[SP_Merge_Sales_Real]
|
||||
@Loadid INTEGER
|
||||
|
|
@ -696,14 +753,14 @@ WHERE
|
|||
self.validate_all(
|
||||
"SELECT CONVERT(VARCHAR(10), testdb.dbo.test.x, 120) y FROM testdb.dbo.test",
|
||||
write={
|
||||
"mysql": "SELECT CAST(DATE_FORMAT(testdb.dbo.test.x, '%Y-%m-%d %T') AS VARCHAR(10)) AS y FROM testdb.dbo.test",
|
||||
"mysql": "SELECT CAST(DATE_FORMAT(testdb.dbo.test.x, '%Y-%m-%d %T') AS CHAR(10)) AS y FROM testdb.dbo.test",
|
||||
"spark": "SELECT CAST(DATE_FORMAT(testdb.dbo.test.x, 'yyyy-MM-dd HH:mm:ss') AS VARCHAR(10)) AS y FROM testdb.dbo.test",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT CONVERT(VARCHAR(10), y.x) z FROM testdb.dbo.test y",
|
||||
write={
|
||||
"mysql": "SELECT CAST(y.x AS VARCHAR(10)) AS z FROM testdb.dbo.test AS y",
|
||||
"mysql": "SELECT CAST(y.x AS CHAR(10)) AS z FROM testdb.dbo.test AS y",
|
||||
"spark": "SELECT CAST(y.x AS VARCHAR(10)) AS z FROM testdb.dbo.test AS y",
|
||||
},
|
||||
)
|
||||
|
|
@ -838,7 +895,8 @@ WHERE
|
|||
write={"spark": "SELECT FORMAT_NUMBER(1000000.01, '###,###.###')"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT FORMAT(1234567, 'f')", write={"spark": "SELECT FORMAT_NUMBER(1234567, 'f')"}
|
||||
"SELECT FORMAT(1234567, 'f')",
|
||||
write={"spark": "SELECT FORMAT_NUMBER(1234567, 'f')"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT FORMAT('01-01-1991', 'dd.mm.yyyy')",
|
||||
|
|
@ -853,7 +911,8 @@ WHERE
|
|||
write={"spark": "SELECT DATE_FORMAT(date_col, 'MMMM d')"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT FORMAT(num_col, 'c')", write={"spark": "SELECT FORMAT_NUMBER(num_col, 'c')"}
|
||||
"SELECT FORMAT(num_col, 'c')",
|
||||
write={"spark": "SELECT FORMAT_NUMBER(num_col, 'c')"},
|
||||
)
|
||||
|
||||
def test_string(self):
|
||||
|
|
|
|||
3
tests/fixtures/identity.sql
vendored
3
tests/fixtures/identity.sql
vendored
|
|
@ -76,6 +76,7 @@ map.x
|
|||
SELECT update
|
||||
SELECT x.update
|
||||
SELECT call.x
|
||||
SELECT end
|
||||
a.b.INT(1.234)
|
||||
INT(x / 100)
|
||||
time * 100
|
||||
|
|
@ -269,6 +270,8 @@ SELECT x LIKE '%x%' FROM test
|
|||
SELECT * FROM test LIMIT 100
|
||||
SELECT * FROM test LIMIT 1 + 1
|
||||
SELECT * FROM test LIMIT 100 OFFSET 200
|
||||
SELECT * FROM test LIMIT (SELECT 1)
|
||||
SELECT * FROM test LIMIT (SELECT 1) OFFSET (SELECT 1)
|
||||
SELECT * FROM test FETCH FIRST ROWS ONLY
|
||||
SELECT * FROM test FETCH FIRST 1 ROWS ONLY
|
||||
SELECT * FROM test ORDER BY id DESC FETCH FIRST 10 ROWS WITH TIES
|
||||
|
|
|
|||
|
|
@ -93,3 +93,7 @@ WITH cte2 AS (SELECT a FROM x), cte1 AS (SELECT t.a FROM cte2 AS t) SELECT a FRO
|
|||
-- Duplicate CTE nested in CTE
|
||||
WITH cte1 AS (SELECT a FROM x), cte2 AS (WITH cte3 AS (SELECT a FROM x) SELECT a FROM cte3) SELECT a FROM cte2;
|
||||
WITH cte1 AS (SELECT a FROM x), cte2 AS (SELECT a FROM cte1 AS cte3) SELECT a FROM cte2;
|
||||
|
||||
-- Wrapped subquery joined with table
|
||||
SELECT * FROM ((SELECT c FROM t1) JOIN t2);
|
||||
WITH cte AS (SELECT c FROM t1) SELECT * FROM (cte AS cte, t2);
|
||||
|
|
|
|||
24
tests/fixtures/optimizer/merge_subqueries.sql
vendored
24
tests/fixtures/optimizer/merge_subqueries.sql
vendored
|
|
@ -372,3 +372,27 @@ FROM x AS x
|
|||
LEFT JOIN i AS i
|
||||
ON x.a = i.a;
|
||||
WITH i AS (SELECT x.a AS a FROM y AS y JOIN x AS x ON y.b = x.b) SELECT x.a AS a FROM x AS x LEFT JOIN i AS i ON x.a = i.a;
|
||||
|
||||
# title: Outer scope selects from wrapped table with a join (unknown schema)
|
||||
# execute: false
|
||||
WITH _q_0 AS (SELECT t1.c AS c FROM t1 AS t1) SELECT * FROM (_q_0 AS _q_0 CROSS JOIN t2 AS t2);
|
||||
WITH _q_0 AS (SELECT t1.c AS c FROM t1 AS t1) SELECT * FROM (_q_0 AS _q_0 CROSS JOIN t2 AS t2);
|
||||
|
||||
# title: Outer scope selects single column from wrapped table with a join
|
||||
WITH _q_0 AS (
|
||||
SELECT
|
||||
x.a AS a
|
||||
FROM x AS x
|
||||
), y_2 AS (
|
||||
SELECT
|
||||
y.b AS b
|
||||
FROM y AS y
|
||||
)
|
||||
SELECT
|
||||
y.b AS b
|
||||
FROM (
|
||||
_q_0 AS _q_0
|
||||
JOIN y_2 AS y
|
||||
ON _q_0.a = y.b
|
||||
);
|
||||
SELECT y.b AS b FROM (x AS x JOIN y AS y ON x.a = y.b);
|
||||
|
|
|
|||
84
tests/fixtures/optimizer/optimizer.sql
vendored
84
tests/fixtures/optimizer/optimizer.sql
vendored
|
|
@ -806,3 +806,87 @@ LEFT JOIN (
|
|||
ON "y"."c" = "z"."c"
|
||||
)
|
||||
ON "x"."b" = "y"."b";
|
||||
|
||||
# title: select * from wrapped subquery
|
||||
# execute: false
|
||||
SELECT * FROM ((SELECT * FROM tbl));
|
||||
WITH "_q_0" AS (
|
||||
SELECT
|
||||
*
|
||||
FROM "tbl" AS "tbl"
|
||||
)
|
||||
SELECT
|
||||
*
|
||||
FROM (
|
||||
"_q_0" AS "_q_0"
|
||||
);
|
||||
|
||||
# title: select * from wrapped subquery joined to a table (known schema)
|
||||
SELECT * FROM ((SELECT * FROM x) INNER JOIN y ON a = c);
|
||||
SELECT
|
||||
"x"."a" AS "a",
|
||||
"x"."b" AS "b",
|
||||
"y"."b" AS "b",
|
||||
"y"."c" AS "c"
|
||||
FROM (
|
||||
"x" AS "x"
|
||||
JOIN "y" AS "y"
|
||||
ON "x"."a" = "y"."c"
|
||||
);
|
||||
|
||||
# title: select * from wrapped subquery joined to a table (unknown schema)
|
||||
# execute: false
|
||||
SELECT * FROM ((SELECT c FROM t1) JOIN t2);
|
||||
WITH "_q_0" AS (
|
||||
SELECT
|
||||
"t1"."c" AS "c"
|
||||
FROM "t1" AS "t1"
|
||||
)
|
||||
SELECT
|
||||
*
|
||||
FROM (
|
||||
"_q_0" AS "_q_0"
|
||||
CROSS JOIN "t2" AS "t2"
|
||||
);
|
||||
|
||||
# title: select specific columns from wrapped subquery joined to a table
|
||||
SELECT b FROM ((SELECT a FROM x) INNER JOIN y ON a = b);
|
||||
SELECT
|
||||
"y"."b" AS "b"
|
||||
FROM (
|
||||
"x" AS "x"
|
||||
JOIN "y" AS "y"
|
||||
ON "x"."a" = "y"."b"
|
||||
);
|
||||
|
||||
# title: select * from wrapped join of subqueries (unknown schema)
|
||||
# execute: false
|
||||
SELECT * FROM ((SELECT * FROM t1) JOIN (SELECT * FROM t2));
|
||||
WITH "_q_0" AS (
|
||||
SELECT
|
||||
*
|
||||
FROM "t1" AS "t1"
|
||||
), "_q_1" AS (
|
||||
SELECT
|
||||
*
|
||||
FROM "t2" AS "t2"
|
||||
)
|
||||
SELECT
|
||||
*
|
||||
FROM (
|
||||
"_q_0" AS "_q_0"
|
||||
CROSS JOIN "_q_1" AS "_q_1"
|
||||
);
|
||||
|
||||
# title: select * from wrapped join of subqueries (known schema)
|
||||
SELECT * FROM ((SELECT * FROM x) INNER JOIN (SELECT * FROM y) ON a = c);
|
||||
SELECT
|
||||
"x"."a" AS "a",
|
||||
"x"."b" AS "b",
|
||||
"y"."b" AS "b",
|
||||
"y"."c" AS "c"
|
||||
FROM (
|
||||
"x" AS "x"
|
||||
JOIN "y" AS "y"
|
||||
ON "x"."a" = "y"."c"
|
||||
);
|
||||
|
|
|
|||
|
|
@ -16,9 +16,6 @@ SELECT x.a AS a FROM x AS x JOIN (SELECT y.b AS b FROM y AS y) AS z ON x.b = z.b
|
|||
SELECT x1.a FROM (SELECT * FROM x) AS x1, (SELECT * FROM x) AS x2;
|
||||
SELECT x1.a AS a FROM (SELECT x.a AS a FROM x AS x) AS x1, (SELECT 1 AS _ FROM x AS x) AS x2;
|
||||
|
||||
SELECT x1.a FROM (SELECT * FROM x) AS x1, (SELECT * FROM x) AS x2;
|
||||
SELECT x1.a AS a FROM (SELECT x.a AS a FROM x AS x) AS x1, (SELECT 1 AS _ FROM x AS x) AS x2;
|
||||
|
||||
SELECT a FROM (SELECT DISTINCT a, b FROM x);
|
||||
SELECT _q_0.a AS a FROM (SELECT DISTINCT x.a AS a, x.b AS b FROM x AS x) AS _q_0;
|
||||
|
||||
|
|
@ -77,9 +74,6 @@ SELECT _q_0.a AS a FROM (SELECT aa.a AS a FROM aa AS aa UNION ALL SELECT bb.a AS
|
|||
SELECT a FROM (SELECT a FROM aa UNION ALL SELECT * FROM bb UNION ALL SELECT * from cc);
|
||||
SELECT _q_0.a AS a FROM (SELECT aa.a AS a FROM aa AS aa UNION ALL SELECT bb.a AS a FROM bb AS bb UNION ALL SELECT cc.a AS a FROM cc AS cc) AS _q_0;
|
||||
|
||||
SELECT a FROM (SELECT * FROM aa UNION ALL SELECT * FROM bb UNION ALL SELECT * from cc);
|
||||
SELECT _q_0.a AS a FROM (SELECT aa.a AS a FROM aa AS aa UNION ALL SELECT bb.a AS a FROM bb AS bb UNION ALL SELECT cc.a AS a FROM cc AS cc) AS _q_0;
|
||||
|
||||
SELECT a FROM (SELECT * FROM aa CROSS JOIN bb);
|
||||
SELECT _q_0.a AS a FROM (SELECT a AS a FROM aa AS aa CROSS JOIN bb AS bb) AS _q_0;
|
||||
|
||||
|
|
@ -91,3 +85,6 @@ SELECT _q_1.a AS a FROM (SELECT _q_0.a AS a FROM (SELECT aa.a AS a FROM aa AS aa
|
|||
|
||||
with cte1 as (SELECT cola, colb FROM tb UNION ALL SELECT colc, cold FROM tb2) SELECT cola FROM cte1;
|
||||
WITH cte1 AS (SELECT tb.cola AS cola FROM tb AS tb UNION ALL SELECT tb2.colc AS colc FROM tb2 AS tb2) SELECT cte1.cola AS cola FROM cte1;
|
||||
|
||||
SELECT * FROM ((SELECT c FROM t1) JOIN t2);
|
||||
SELECT * FROM ((SELECT t1.c AS c FROM t1 AS t1) AS _q_0, t2 AS t2);
|
||||
|
|
|
|||
44
tests/fixtures/optimizer/qualify_columns.sql
vendored
44
tests/fixtures/optimizer/qualify_columns.sql
vendored
|
|
@ -307,6 +307,11 @@ SELECT x.b AS b, y.b AS b, y.c AS c FROM x AS x, y AS y;
|
|||
SELECT * EXCEPT(a) FROM x;
|
||||
SELECT x.b AS b FROM x AS x;
|
||||
|
||||
# execute: false
|
||||
# note: this query would fail in the engine level because there are 0 selected columns
|
||||
SELECT * EXCEPT (a, b) FROM x;
|
||||
SELECT * EXCEPT (x.a, x.b) FROM x AS x;
|
||||
|
||||
--------------------------------------
|
||||
-- Using
|
||||
--------------------------------------
|
||||
|
|
@ -410,8 +415,7 @@ SELECT _q_0.c AS c FROM (SELECT 1 AS a) AS x LATERAL VIEW EXPLODE(x.a) _q_0 AS c
|
|||
--------------------------------------
|
||||
-- Window functions
|
||||
--------------------------------------
|
||||
|
||||
-- ORDER BY in window function
|
||||
# title: ORDER BY in window function
|
||||
SELECT a + 1 AS a, ROW_NUMBER() OVER (PARTITION BY b ORDER BY a) AS row_num FROM x;
|
||||
SELECT x.a + 1 AS a, ROW_NUMBER() OVER (PARTITION BY x.b ORDER BY x.a) AS row_num FROM x AS x;
|
||||
|
||||
|
|
@ -429,7 +433,6 @@ SELECT x.a AS a, x.b AS b FROM x AS x QUALIFY COUNT(x.a) OVER (PARTITION BY x.b)
|
|||
--------------------------------------
|
||||
-- Expand laterals
|
||||
--------------------------------------
|
||||
|
||||
# execute: false
|
||||
select 2 AS d, d + 1 FROM x WHERE d = 2 GROUP BY d;
|
||||
SELECT 2 AS d, 2 + 1 AS _col_1 FROM x AS x WHERE 2 = 2 GROUP BY 1;
|
||||
|
|
@ -457,3 +460,38 @@ FROM (
|
|||
FROM x
|
||||
);
|
||||
SELECT _q_0.i AS i, _q_0.j AS j FROM (SELECT x.a + 1 AS i, x.a + 1 + 1 AS j FROM x AS x) AS _q_0;
|
||||
|
||||
--------------------------------------
|
||||
-- Wrapped tables / join constructs
|
||||
--------------------------------------
|
||||
# execute: false
|
||||
SELECT * FROM ((tbl));
|
||||
SELECT * FROM ((tbl AS tbl));
|
||||
|
||||
SELECT a, c FROM (x LEFT JOIN y ON a = c);
|
||||
SELECT x.a AS a, y.c AS c FROM (x AS x LEFT JOIN y AS y ON x.a = y.c);
|
||||
|
||||
# execute: false
|
||||
SELECT * FROM ((a CROSS JOIN ((b CROSS JOIN c) CROSS JOIN (d CROSS JOIN e))));
|
||||
SELECT * FROM ((a AS a CROSS JOIN ((b AS b CROSS JOIN c AS c) CROSS JOIN (d AS d CROSS JOIN e AS e))));
|
||||
|
||||
# execute: false
|
||||
SELECT * FROM ((SELECT * FROM tbl));
|
||||
SELECT * FROM ((SELECT * FROM tbl AS tbl) AS _q_0);
|
||||
|
||||
# execute: false
|
||||
SELECT * FROM ((SELECT c FROM t1) JOIN t2);
|
||||
SELECT * FROM ((SELECT t1.c AS c FROM t1 AS t1) AS _q_0, t2 AS t2);
|
||||
|
||||
# execute: false
|
||||
SELECT * FROM ((SELECT * FROM x) INNER JOIN y ON a = c);
|
||||
SELECT y.b AS b, y.c AS c, _q_0.a AS a, _q_0.b AS b FROM ((SELECT x.a AS a, x.b AS b FROM x AS x) AS _q_0 INNER JOIN y AS y ON _q_0.a = y.c);
|
||||
|
||||
SELECT x.a, y.b, z.c FROM x LEFT JOIN (y INNER JOIN z ON y.c = z.c) ON x.b = y.b;
|
||||
SELECT x.a AS a, y.b AS b, z.c AS c FROM x AS x LEFT JOIN (y AS y INNER JOIN z AS z ON y.c = z.c) ON x.b = y.b;
|
||||
|
||||
SELECT * FROM ((SELECT * FROM x) INNER JOIN (SELECT * FROM y) ON a = c);
|
||||
SELECT _q_0.a AS a, _q_0.b AS b, _q_1.b AS b, _q_1.c AS c FROM ((SELECT x.a AS a, x.b AS b FROM x AS x) AS _q_0 INNER JOIN (SELECT y.b AS b, y.c AS c FROM y AS y) AS _q_1 ON _q_0.a = _q_1.c);
|
||||
|
||||
SELECT b FROM ((SELECT a FROM x) INNER JOIN y ON a = b);
|
||||
SELECT y.b AS b FROM ((SELECT x.a AS a FROM x AS x) AS _q_0 INNER JOIN y AS y ON _q_0.a = y.b);
|
||||
|
|
|
|||
|
|
@ -10,4 +10,4 @@ SELECT b FROM x AS a CROSS JOIN y AS b CROSS JOIN y AS c;
|
|||
SELECT x.a FROM x JOIN y USING (a);
|
||||
SELECT a, SUM(b) FROM x GROUP BY 3;
|
||||
SELECT p FROM (SELECT x from xx) y CROSS JOIN yy CROSS JOIN zz
|
||||
select a from (select * from x cross join y);
|
||||
SELECT a FROM (SELECT * FROM x CROSS JOIN y);
|
||||
|
|
|
|||
8
tests/fixtures/optimizer/qualify_tables.sql
vendored
8
tests/fixtures/optimizer/qualify_tables.sql
vendored
|
|
@ -101,3 +101,11 @@ SELECT * FROM (SELECT * FROM c.db.tbl1 AS tbl1 JOIN c.db.tbl2 AS tbl2 ON id1 = i
|
|||
# title: join construct within join construct
|
||||
SELECT * FROM (tbl1 AS tbl1 JOIN (tbl2 AS tbl2 JOIN tbl3 AS tbl3 ON id2 = id3) AS _q_0 ON id1 = id3) AS _q_1;
|
||||
SELECT * FROM (SELECT * FROM c.db.tbl1 AS tbl1 JOIN (SELECT * FROM c.db.tbl2 AS tbl2 JOIN c.db.tbl3 AS tbl3 ON id2 = id3) AS _q_0 ON id1 = id3) AS _q_1;
|
||||
|
||||
# title: wrapped subquery without alias
|
||||
SELECT * FROM ((SELECT * FROM t));
|
||||
SELECT * FROM ((SELECT * FROM c.db.t AS t) AS _q_0);
|
||||
|
||||
# title: wrapped subquery without alias joined with a table
|
||||
SELECT * FROM ((SELECT * FROM t1) INNER JOIN t2 ON a = b);
|
||||
SELECT * FROM ((SELECT * FROM c.db.t1 AS t1) AS _q_0 INNER JOIN c.db.t2 AS t2 ON a = b);
|
||||
|
|
|
|||
|
|
@ -58,6 +58,9 @@ class TestBuild(unittest.TestCase):
|
|||
(lambda: x.as_("y"), "x AS y"),
|
||||
(lambda: x.isin(1, "2"), "x IN (1, '2')"),
|
||||
(lambda: x.isin(query="select 1"), "x IN (SELECT 1)"),
|
||||
(lambda: x.isin(unnest="x"), "x IN (SELECT UNNEST(x))"),
|
||||
(lambda: x.isin(unnest="x"), "x IN UNNEST(x)", "bigquery"),
|
||||
(lambda: x.isin(unnest=["x", "y"]), "x IN (SELECT UNNEST(x, y))"),
|
||||
(lambda: x.between(1, 2), "x BETWEEN 1 AND 2"),
|
||||
(lambda: 1 + x + 2 + 3, "1 + x + 2 + 3"),
|
||||
(lambda: 1 + x * 2 + 3, "1 + (x * 2) + 3"),
|
||||
|
|
|
|||
|
|
@ -556,6 +556,7 @@ class TestExpressions(unittest.TestCase):
|
|||
self.assertIsInstance(parse_one("HEX(foo)"), exp.Hex)
|
||||
self.assertIsInstance(parse_one("TO_HEX(foo)", read="bigquery"), exp.Hex)
|
||||
self.assertIsInstance(parse_one("TO_HEX(MD5(foo))", read="bigquery"), exp.MD5)
|
||||
self.assertIsInstance(parse_one("TRANSFORM(a, b)", read="spark"), exp.Transform)
|
||||
|
||||
def test_column(self):
|
||||
column = parse_one("a.b.c.d")
|
||||
|
|
|
|||
|
|
@ -186,3 +186,16 @@ class TestLineage(unittest.TestCase):
|
|||
self.assertEqual(downstream.alias, "")
|
||||
|
||||
self.assertEqual(downstream.downstream, [])
|
||||
|
||||
def test_lineage_union(self) -> None:
|
||||
node = lineage(
|
||||
"x",
|
||||
"SELECT ax AS x FROM a UNION SELECT bx FROM b UNION SELECT cx FROM c",
|
||||
)
|
||||
assert len(node.downstream) == 3
|
||||
|
||||
node = lineage(
|
||||
"x",
|
||||
"SELECT x FROM (SELECT ax AS x FROM a UNION SELECT bx FROM b UNION SELECT cx FROM c)",
|
||||
)
|
||||
assert len(node.downstream) == 3
|
||||
|
|
|
|||
|
|
@ -427,6 +427,27 @@ FROM READ_CSV('tests/fixtures/optimizer/tpc-h/nation.csv.gz', 'delimiter', '|')
|
|||
{"s.b"},
|
||||
)
|
||||
|
||||
# Check that parentheses don't introduce a new scope unless an alias is attached
|
||||
sql = "SELECT * FROM (((SELECT * FROM (t1 JOIN t2) AS t3) JOIN (SELECT * FROM t4)))"
|
||||
expression = parse_one(sql)
|
||||
for scopes in traverse_scope(expression), list(build_scope(expression).traverse()):
|
||||
self.assertEqual(len(scopes), 4)
|
||||
|
||||
self.assertEqual(scopes[0].expression.sql(), "t1, t2")
|
||||
self.assertEqual(set(scopes[0].sources), {"t1", "t2"})
|
||||
|
||||
self.assertEqual(scopes[1].expression.sql(), "SELECT * FROM (t1, t2) AS t3")
|
||||
self.assertEqual(set(scopes[1].sources), {"t3"})
|
||||
|
||||
self.assertEqual(scopes[2].expression.sql(), "SELECT * FROM t4")
|
||||
self.assertEqual(set(scopes[2].sources), {"t4"})
|
||||
|
||||
self.assertEqual(
|
||||
scopes[3].expression.sql(),
|
||||
"SELECT * FROM (((SELECT * FROM (t1, t2) AS t3), (SELECT * FROM t4)))",
|
||||
)
|
||||
self.assertEqual(set(scopes[3].sources), {""})
|
||||
|
||||
@patch("sqlglot.optimizer.scope.logger")
|
||||
def test_scope_warning(self, logger):
|
||||
self.assertEqual(len(traverse_scope(parse_one("WITH q AS (@y) SELECT * FROM q"))), 1)
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
import unittest
|
||||
|
||||
from sqlglot.dialects import BigQuery
|
||||
from sqlglot.errors import TokenError
|
||||
from sqlglot.tokens import Tokenizer, TokenType
|
||||
|
||||
|
||||
|
|
@ -65,7 +66,7 @@ x"""
|
|||
self.assertEqual(tokens[3].token_type, TokenType.SEMICOLON)
|
||||
|
||||
def test_error_msg(self):
|
||||
with self.assertRaisesRegex(ValueError, "Error tokenizing 'select /'"):
|
||||
with self.assertRaisesRegex(TokenError, "Error tokenizing 'select /'"):
|
||||
Tokenizer().tokenize("select /*")
|
||||
|
||||
def test_jinja(self):
|
||||
|
|
|
|||
|
|
@ -186,7 +186,7 @@ WHERE
|
|||
*/
|
||||
SELECT
|
||||
tbl.cola /* comment 1 */ + tbl.colb /* comment 2 */,
|
||||
CAST(x AS INT), # comment 3
|
||||
CAST(x AS CHAR), # comment 3
|
||||
y -- comment 4
|
||||
FROM
|
||||
bar /* comment 5 */,
|
||||
|
|
@ -198,7 +198,7 @@ WHERE
|
|||
*/
|
||||
SELECT
|
||||
tbl.cola /* comment 1 */ + tbl.colb /* comment 2 */,
|
||||
CAST(x AS INT), /* comment 3 */
|
||||
CAST(x AS CHAR), /* comment 3 */
|
||||
y /* comment 4 */
|
||||
FROM bar /* comment 5 */, tbl /* comment 6 */""",
|
||||
read="mysql",
|
||||
|
|
@ -211,9 +211,12 @@ FROM bar /* comment 5 */, tbl /* comment 6 */""",
|
|||
-- comment 1
|
||||
AND bar
|
||||
-- comment 2
|
||||
AND bla;
|
||||
AND bla
|
||||
-- comment 3
|
||||
LIMIT 10
|
||||
;
|
||||
""",
|
||||
"SELECT a FROM b WHERE foo AND /* comment 1 */ bar AND /* comment 2 */ bla",
|
||||
"SELECT a FROM b WHERE foo AND /* comment 1 */ bar AND /* comment 2 */ bla LIMIT 10 /* comment 3 */",
|
||||
)
|
||||
self.validate(
|
||||
"""
|
||||
|
|
@ -285,6 +288,47 @@ FROM v""",
|
|||
"SELECT 1 /* hi this is a comment */",
|
||||
read="snowflake",
|
||||
)
|
||||
self.validate(
|
||||
"-- comment\nDROP TABLE IF EXISTS foo",
|
||||
"/* comment */ DROP TABLE IF EXISTS foo",
|
||||
)
|
||||
self.validate(
|
||||
"""
|
||||
-- comment1
|
||||
-- comment2
|
||||
|
||||
-- comment3
|
||||
DROP TABLE IF EXISTS db.tba
|
||||
""",
|
||||
"""/* comment1 */
|
||||
/* comment2 */
|
||||
/* comment3 */
|
||||
DROP TABLE IF EXISTS db.tba""",
|
||||
pretty=True,
|
||||
)
|
||||
self.validate(
|
||||
"""
|
||||
CREATE TABLE db.tba AS
|
||||
SELECT a, b, c
|
||||
FROM tb_01
|
||||
WHERE
|
||||
-- comment5
|
||||
a = 1 AND b = 2 --comment6
|
||||
-- and c = 1
|
||||
-- comment7
|
||||
""",
|
||||
"""CREATE TABLE db.tba AS
|
||||
SELECT
|
||||
a,
|
||||
b,
|
||||
c
|
||||
FROM tb_01
|
||||
WHERE
|
||||
a /* comment5 */ = 1 AND b = 2 /* comment6 */
|
||||
/* and c = 1 */
|
||||
/* comment7 */""",
|
||||
pretty=True,
|
||||
)
|
||||
|
||||
def test_types(self):
|
||||
self.validate("INT 1", "CAST(1 AS INT)")
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue