Adding upstream version 17.4.1.
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
parent
d28371fcc4
commit
5c63f2bde9
GPG key ID:
FBB4F0E80A80222F
78 changed files with 35859 additions and 34717 deletions
41
CHANGELOG.md
41
CHANGELOG.md
|
|
@ -1,6 +1,45 @@
|
|||
Changelog
|
||||
=========
|
||||
|
||||
## [v17.4.0] - 2023-07-11
|
||||
### :boom: BREAKING CHANGES
|
||||
- due to [`c511278`](https://github.com/tobymao/sqlglot/commit/c51127842ffbb5dbe02c33de3ba8e04b69e6ebc3) - table_name force quotes if unsafe identifiers *(commit by [@tobymao](https://github.com/tobymao))*:
|
||||
|
||||
table_name force quotes if unsafe identifiers
|
||||
|
||||
|
||||
### :sparkles: New Features
|
||||
- [`21246fb`](https://github.com/tobymao/sqlglot/commit/21246fbd8e308d8113ee56ad58f542eaad1d823f) - Add FLOAT(n) mappings to Hive *(PR [#1896](https://github.com/tobymao/sqlglot/pull/1896) by [@dmoore247](https://github.com/dmoore247))*
|
||||
- [`d68f844`](https://github.com/tobymao/sqlglot/commit/d68f844bf6f65c933dbf5d155faa3a88e93ec6e7) - **tsql**: insert output closes [#1901](https://github.com/tobymao/sqlglot/pull/1901) *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
- [`273daf9`](https://github.com/tobymao/sqlglot/commit/273daf96b19c2166a3d960e018153ef0748f97c0) - add returning as alias for postgres *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
|
||||
### :bug: Bug Fixes
|
||||
- [`900ad7e`](https://github.com/tobymao/sqlglot/commit/900ad7eddde7bc065e6a6ab46389077cfd43e353) - **mysql**: add support some logical operators *(PR [#1899](https://github.com/tobymao/sqlglot/pull/1899) by [@brosoul](https://github.com/brosoul))*
|
||||
- [`c3b4b66`](https://github.com/tobymao/sqlglot/commit/c3b4b66b765af262b36774ff7205944b34b6e20c) - expansions of literals in select and where *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
- [`6ccb595`](https://github.com/tobymao/sqlglot/commit/6ccb5956568f60d154c66a63050e0edbfc2394ea) - postgres nested jsonb closes [#1903](https://github.com/tobymao/sqlglot/pull/1903) *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
- [`a27521b`](https://github.com/tobymao/sqlglot/commit/a27521bb8a3491de0b85c86af336233c70710f72) - improve parsing / generation of REGEXP_EXTRACT *(PR [#1905](https://github.com/tobymao/sqlglot/pull/1905) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- [`3e8c6cb`](https://github.com/tobymao/sqlglot/commit/3e8c6cb1b27e7f4c3ecad0f9f3a9d6fb6237ba3f) - **oracle**: keep NCHAR as-is *(PR [#1908](https://github.com/tobymao/sqlglot/pull/1908) by [@mpf82](https://github.com/mpf82))*
|
||||
- [`5eed17e`](https://github.com/tobymao/sqlglot/commit/5eed17e394f4698362de32c7e202e5db3b3db486) - use sql method to generate index name *(PR [#1909](https://github.com/tobymao/sqlglot/pull/1909) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- [`c511278`](https://github.com/tobymao/sqlglot/commit/c51127842ffbb5dbe02c33de3ba8e04b69e6ebc3) - table_name force quotes if unsafe identifiers *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
|
||||
### :recycle: Refactors
|
||||
- [`3b215ad`](https://github.com/tobymao/sqlglot/commit/3b215adc413772bc1af46a67e2410b38ad8872a2) - **hive**: cleanup handling of FLOAT(n) in the generator *(commit by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- [`69a69f6`](https://github.com/tobymao/sqlglot/commit/69a69f67d06ebf8d328eb30fd0ef7d717758a4c9) - show an 'unsupported' error when transpiling (+) *(PR [#1906](https://github.com/tobymao/sqlglot/pull/1906) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
|
||||
|
||||
## [v17.3.0] - 2023-07-07
|
||||
### :sparkles: New Features
|
||||
- [`fafccf5`](https://github.com/tobymao/sqlglot/commit/fafccf5a8e21028ef3a678cf989332b4e7db6825) - **postgres**: add MONEY type, revert in Spark to use DECIMAL(15, 4) *(commit by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- [`24dda47`](https://github.com/tobymao/sqlglot/commit/24dda47dd85abab79cd5ea126262a8e00639ca6f) - improve transpilation of BigQuery's TO_HEX(MD5(..)) *(PR [#1897](https://github.com/tobymao/sqlglot/pull/1897) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
|
||||
### :bug: Bug Fixes
|
||||
- [`2ab8887`](https://github.com/tobymao/sqlglot/commit/2ab888786eea5b2c67446c8eb7779b85288bc07b) - **spark**: map MONEY type to a broader DECIMAL type *(commit by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- [`dd7e494`](https://github.com/tobymao/sqlglot/commit/dd7e494c022a724d76fb592bc67c41c18f3c94ec) - enforce function arg order *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
|
||||
### :wrench: Chores
|
||||
- [`5cf9304`](https://github.com/tobymao/sqlglot/commit/5cf930420e0dd774ff871ecc36115caae452389d) - **optimizer**: add more tests, improve titles in qualify_tables.sql *(commit by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
|
||||
|
||||
## [v17.2.0] - 2023-07-06
|
||||
### :bug: Bug Fixes
|
||||
- [`1f575db`](https://github.com/tobymao/sqlglot/commit/1f575db44f9c75c92fd5abf6c514c9933b1175b6) - **duckdb**: improve transpilation of BigQuery DATE function *(PR [#1895](https://github.com/tobymao/sqlglot/pull/1895) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
|
|
@ -818,3 +857,5 @@ Changelog
|
|||
[v17.0.0]: https://github.com/tobymao/sqlglot/compare/v16.8.1...v17.0.0
|
||||
[v17.1.0]: https://github.com/tobymao/sqlglot/compare/v17.0.0...v17.1.0
|
||||
[v17.2.0]: https://github.com/tobymao/sqlglot/compare/v17.1.0...v17.2.0
|
||||
[v17.3.0]: https://github.com/tobymao/sqlglot/compare/v17.2.0...v17.3.0
|
||||
[v17.4.0]: https://github.com/tobymao/sqlglot/compare/v17.3.0...v17.4.0
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -51,8 +51,8 @@
|
|||
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos">1</span></a><span class="c1"># file generated by setuptools_scm</span>
|
||||
</span><span id="L-2"><a href="#L-2"><span class="linenos">2</span></a><span class="c1"># don't change, don't track in version control</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos">3</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">'17.2.0'</span>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos">4</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">17</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos">3</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">'17.4.0'</span>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos">4</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">17</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -777,7 +777,7 @@
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">createDataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">data</span><span class="p">:</span> <span class="n">Sequence</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963616969344'</span><span class="o">></span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963616969344'</span><span class="o">></span><span class="p">],</span> <span class="n">Tuple</span><span class="p">]]</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963617390416'</span><span class="o">></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">samplingRatio</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">verifySchema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
<span class="name">createDataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">data</span><span class="p">:</span> <span class="n">Sequence</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904735552'</span><span class="o">></span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904735552'</span><span class="o">></span><span class="p">],</span> <span class="n">Tuple</span><span class="p">]]</span>,</span><span class="param"> <span class="n">schema</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411906690064'</span><span class="o">></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">samplingRatio</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">float</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">verifySchema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="SparkSession.createDataFrame-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -1681,7 +1681,7 @@
|
|||
<input id="DataFrame.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
|
||||
<span class="name">DataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">spark</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963621507328'</span><span class="o">></span>,</span><span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Select">sqlglot.expressions.Select</a></span>,</span><span class="param"> <span class="n">branch_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">sequence_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">last_op</span><span class="p">:</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">dataframe</span><span class="o">.</span><span class="n">sql</span><span class="o">.</span><span class="n">operations</span><span class="o">.</span><span class="n">Operation</span> <span class="o">=</span> <span class="o"><</span><span class="n">Operation</span><span class="o">.</span><span class="n">INIT</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span><span class="o">></span>,</span><span class="param"> <span class="n">pending_hints</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">output_expression_container</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963621708112'</span><span class="o">></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span>)</span>
|
||||
<span class="name">DataFrame</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">spark</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411909715808'</span><span class="o">></span>,</span><span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Select">sqlglot.expressions.Select</a></span>,</span><span class="param"> <span class="n">branch_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">sequence_id</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">last_op</span><span class="p">:</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">dataframe</span><span class="o">.</span><span class="n">sql</span><span class="o">.</span><span class="n">operations</span><span class="o">.</span><span class="n">Operation</span> <span class="o">=</span> <span class="o"><</span><span class="n">Operation</span><span class="o">.</span><span class="n">INIT</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span><span class="o">></span>,</span><span class="param"> <span class="n">pending_hints</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">output_expression_container</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411909885280'</span><span class="o">></span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span>)</span>
|
||||
|
||||
<label class="view-source-button" for="DataFrame.__init__-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -2611,7 +2611,7 @@ is unlikely to come up.</p>
|
|||
<div class="decorator">@operation(Operation.FROM)</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">fillna</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963617155808'</span><span class="o">></span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
<span class="name">fillna</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411905248896'</span><span class="o">></span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="DataFrame.fillna-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -2680,7 +2680,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="decorator">@operation(Operation.FROM)</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">replace</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">to_replace</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">Dict</span><span class="p">]</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n">Collection</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963615469216'</span><span class="o">></span><span class="p">],</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963615469216'</span><span class="o">></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
<span class="name">replace</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">to_replace</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">Dict</span><span class="p">]</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">List</span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>,</span><span class="param"> <span class="n">subset</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="n">Collection</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903718352'</span><span class="o">></span><span class="p">],</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903718352'</span><span class="o">></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="DataFrame.replace-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -2885,7 +2885,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="decorator">@operation(Operation.NO_OP)</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">repartition</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">numPartitions</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963615678464'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963615757024'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
<span class="name">repartition</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">numPartitions</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903922128'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904015376'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#DataFrame">sqlglot.dataframe.sql.DataFrame</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="DataFrame.repartition-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -3590,7 +3590,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<input id="Column.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
|
||||
<span class="name">Column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963617714832'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span>)</span>
|
||||
<span class="name">Column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411907807680'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span>)</span>
|
||||
|
||||
<label class="view-source-button" for="Column.__init__-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -3633,7 +3633,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">ensure_col</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963615805936'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span><span class="return-annotation">):</span></span>
|
||||
<span class="name">ensure_col</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">value</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904110128'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">,</span> <span class="n">NoneType</span><span class="p">]</span></span><span class="return-annotation">):</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.ensure_col-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -3654,7 +3654,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">ensure_cols</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">args</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963616303168'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span><span class="p">]</span>:</span></span>
|
||||
<span class="name">ensure_cols</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">args</span><span class="p">:</span> <span class="n">List</span><span class="p">[</span><span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904283728'</span><span class="o">></span><span class="p">,</span> <span class="n"><a href="../expressions.html#Expression">sqlglot.expressions.Expression</a></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n">List</span><span class="p">[</span><span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span><span class="p">]</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.ensure_cols-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -3675,7 +3675,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">invoke_anonymous_function</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963616124384'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="n">func_name</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963616115248'</span><span class="o">></span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
<span class="name">invoke_anonymous_function</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903793648'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="n">func_name</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="o">*</span><span class="n">args</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904390720'</span><span class="o">></span><span class="p">]</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.invoke_anonymous_function-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -3702,7 +3702,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">invoke_expression_over_column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963615935648'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="n">callable_expression</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
<span class="name">invoke_expression_over_column</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="n">column</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904276080'</span><span class="o">></span><span class="p">]</span>,</span><span class="param"> <span class="n">callable_expression</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.invoke_expression_over_column-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -3739,7 +3739,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963616221392'</span><span class="o">></span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
<span class="name">binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904480480'</span><span class="o">></span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.binary_op-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -3760,7 +3760,7 @@ and check if it matches the type of the value provided. If not then make it null
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">inverse_binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963616249648'</span><span class="o">></span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
<span class="name">inverse_binary_op</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">klass</span><span class="p">:</span> <span class="n">Callable</span>,</span><span class="param"> <span class="n">other</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411904268496'</span><span class="o">></span>,</span><span class="param"> <span class="o">**</span><span class="n">kwargs</span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.inverse_binary_op-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -4314,7 +4314,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">isin</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963616428672'</span><span class="o">></span><span class="p">,</span> <span class="n">Iterable</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963616428672'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">):</span></span>
|
||||
<span class="name">isin</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902623952'</span><span class="o">></span><span class="p">,</span> <span class="n">Iterable</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902623952'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">):</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.isin-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -4335,7 +4335,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">between</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">lowerBound</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963614455280'</span><span class="o">></span>,</span><span class="param"> <span class="n">upperBound</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963614509760'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
<span class="name">between</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">lowerBound</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902714368'</span><span class="o">></span>,</span><span class="param"> <span class="n">upperBound</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902752464'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.between-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -4370,7 +4370,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">over</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">window</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963614567472'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
<span class="name">over</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">window</span><span class="p">:</span> <span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902826560'</span><span class="o">></span></span><span class="return-annotation">) -> <span class="n"><a href="#Column">sqlglot.dataframe.sql.Column</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Column.over-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -4615,7 +4615,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
|||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963615316912'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963615316912'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903155680'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903155680'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Window.partitionBy-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -4636,7 +4636,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
|||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963614725072'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963614725072'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">cls</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903260272'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411903260272'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Window.orderBy-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -4872,7 +4872,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963614810736'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963614810736'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
<span class="name">partitionBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902905984'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902905984'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="WindowSpec.partitionBy-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
@ -4899,7 +4899,7 @@ Sqlglot doesn't currently replicate this class so it only accepts a string</p>
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963614775808'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'139963614775808'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
<span class="name">orderBy</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="o">*</span><span class="n">cols</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902894160'</span><span class="o">></span><span class="p">,</span> <span class="n">List</span><span class="p">[</span><span class="o"><</span><span class="n">MagicMock</span> <span class="nb">id</span><span class="o">=</span><span class="s1">'140411902894160'</span><span class="o">></span><span class="p">]]</span></span><span class="return-annotation">) -> <span class="n"><a href="#WindowSpec">sqlglot.dataframe.sql.WindowSpec</a></span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="WindowSpec.orderBy-view-source"><span>View Source</span></label>
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -354,6 +354,9 @@
|
|||
<li>
|
||||
<a class="function" href="#concat_to_dpipe_sql">concat_to_dpipe_sql</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="function" href="#regexp_extract_sql">regexp_extract_sql</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="function" href="#pivot_column_names">pivot_column_names</a>
|
||||
</li>
|
||||
|
|
@ -1002,27 +1005,36 @@
|
|||
</span><span id="L-620"><a href="#L-620"><span class="linenos">620</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="L-621"><a href="#L-621"><span class="linenos">621</span></a>
|
||||
</span><span id="L-622"><a href="#L-622"><span class="linenos">622</span></a>
|
||||
</span><span id="L-623"><a href="#L-623"><span class="linenos">623</span></a><span class="c1"># Spark, DuckDB use (almost) the same naming scheme for the output columns of the PIVOT operator</span>
|
||||
</span><span id="L-624"><a href="#L-624"><span class="linenos">624</span></a><span class="k">def</span> <span class="nf">pivot_column_names</span><span class="p">(</span><span class="n">aggregations</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
||||
</span><span id="L-625"><a href="#L-625"><span class="linenos">625</span></a> <span class="n">names</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-626"><a href="#L-626"><span class="linenos">626</span></a> <span class="k">for</span> <span class="n">agg</span> <span class="ow">in</span> <span class="n">aggregations</span><span class="p">:</span>
|
||||
</span><span id="L-627"><a href="#L-627"><span class="linenos">627</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">agg</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="L-628"><a href="#L-628"><span class="linenos">628</span></a> <span class="n">names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">agg</span><span class="o">.</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="L-629"><a href="#L-629"><span class="linenos">629</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-630"><a href="#L-630"><span class="linenos">630</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-631"><a href="#L-631"><span class="linenos">631</span></a><span class="sd"> This case corresponds to aggregations without aliases being used as suffixes</span>
|
||||
</span><span id="L-632"><a href="#L-632"><span class="linenos">632</span></a><span class="sd"> (e.g. col_avg(foo)). We need to unquote identifiers because they're going to</span>
|
||||
</span><span id="L-633"><a href="#L-633"><span class="linenos">633</span></a><span class="sd"> be quoted in the base parser's `_parse_pivot` method, due to `to_identifier`.</span>
|
||||
</span><span id="L-634"><a href="#L-634"><span class="linenos">634</span></a><span class="sd"> Otherwise, we'd end up with `col_avg(`foo`)` (notice the double quotes).</span>
|
||||
</span><span id="L-635"><a href="#L-635"><span class="linenos">635</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-636"><a href="#L-636"><span class="linenos">636</span></a> <span class="n">agg_all_unquoted</span> <span class="o">=</span> <span class="n">agg</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
|
||||
</span><span id="L-637"><a href="#L-637"><span class="linenos">637</span></a> <span class="k">lambda</span> <span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">quoted</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-638"><a href="#L-638"><span class="linenos">638</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span>
|
||||
</span><span id="L-639"><a href="#L-639"><span class="linenos">639</span></a> <span class="k">else</span> <span class="n">node</span>
|
||||
</span><span id="L-640"><a href="#L-640"><span class="linenos">640</span></a> <span class="p">)</span>
|
||||
</span><span id="L-641"><a href="#L-641"><span class="linenos">641</span></a> <span class="n">names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">agg_all_unquoted</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">normalize_functions</span><span class="o">=</span><span class="s2">"lower"</span><span class="p">))</span>
|
||||
</span><span id="L-642"><a href="#L-642"><span class="linenos">642</span></a>
|
||||
</span><span id="L-643"><a href="#L-643"><span class="linenos">643</span></a> <span class="k">return</span> <span class="n">names</span>
|
||||
</span><span id="L-623"><a href="#L-623"><span class="linenos">623</span></a><span class="k">def</span> <span class="nf">regexp_extract_sql</span><span class="p">(</span><span class="bp">self</span><span class="p">:</span> <span class="n">Generator</span><span class="p">,</span> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">RegexpExtract</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="L-624"><a href="#L-624"><span class="linenos">624</span></a> <span class="n">bad_args</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">,</span> <span class="p">(</span><span class="s2">"position"</span><span class="p">,</span> <span class="s2">"occurrence"</span><span class="p">,</span> <span class="s2">"parameters"</span><span class="p">)))</span>
|
||||
</span><span id="L-625"><a href="#L-625"><span class="linenos">625</span></a> <span class="k">if</span> <span class="n">bad_args</span><span class="p">:</span>
|
||||
</span><span id="L-626"><a href="#L-626"><span class="linenos">626</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">unsupported</span><span class="p">(</span><span class="sa">f</span><span class="s2">"REGEXP_EXTRACT does not support the following arg(s): </span><span class="si">{</span><span class="n">bad_args</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="L-627"><a href="#L-627"><span class="linenos">627</span></a>
|
||||
</span><span id="L-628"><a href="#L-628"><span class="linenos">628</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">func</span><span class="p">(</span>
|
||||
</span><span id="L-629"><a href="#L-629"><span class="linenos">629</span></a> <span class="s2">"REGEXP_EXTRACT"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"group"</span><span class="p">)</span>
|
||||
</span><span id="L-630"><a href="#L-630"><span class="linenos">630</span></a> <span class="p">)</span>
|
||||
</span><span id="L-631"><a href="#L-631"><span class="linenos">631</span></a>
|
||||
</span><span id="L-632"><a href="#L-632"><span class="linenos">632</span></a>
|
||||
</span><span id="L-633"><a href="#L-633"><span class="linenos">633</span></a><span class="k">def</span> <span class="nf">pivot_column_names</span><span class="p">(</span><span class="n">aggregations</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
||||
</span><span id="L-634"><a href="#L-634"><span class="linenos">634</span></a> <span class="n">names</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-635"><a href="#L-635"><span class="linenos">635</span></a> <span class="k">for</span> <span class="n">agg</span> <span class="ow">in</span> <span class="n">aggregations</span><span class="p">:</span>
|
||||
</span><span id="L-636"><a href="#L-636"><span class="linenos">636</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">agg</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="L-637"><a href="#L-637"><span class="linenos">637</span></a> <span class="n">names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">agg</span><span class="o">.</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="L-638"><a href="#L-638"><span class="linenos">638</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-639"><a href="#L-639"><span class="linenos">639</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-640"><a href="#L-640"><span class="linenos">640</span></a><span class="sd"> This case corresponds to aggregations without aliases being used as suffixes</span>
|
||||
</span><span id="L-641"><a href="#L-641"><span class="linenos">641</span></a><span class="sd"> (e.g. col_avg(foo)). We need to unquote identifiers because they're going to</span>
|
||||
</span><span id="L-642"><a href="#L-642"><span class="linenos">642</span></a><span class="sd"> be quoted in the base parser's `_parse_pivot` method, due to `to_identifier`.</span>
|
||||
</span><span id="L-643"><a href="#L-643"><span class="linenos">643</span></a><span class="sd"> Otherwise, we'd end up with `col_avg(`foo`)` (notice the double quotes).</span>
|
||||
</span><span id="L-644"><a href="#L-644"><span class="linenos">644</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-645"><a href="#L-645"><span class="linenos">645</span></a> <span class="n">agg_all_unquoted</span> <span class="o">=</span> <span class="n">agg</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
|
||||
</span><span id="L-646"><a href="#L-646"><span class="linenos">646</span></a> <span class="k">lambda</span> <span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">quoted</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-647"><a href="#L-647"><span class="linenos">647</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span>
|
||||
</span><span id="L-648"><a href="#L-648"><span class="linenos">648</span></a> <span class="k">else</span> <span class="n">node</span>
|
||||
</span><span id="L-649"><a href="#L-649"><span class="linenos">649</span></a> <span class="p">)</span>
|
||||
</span><span id="L-650"><a href="#L-650"><span class="linenos">650</span></a> <span class="n">names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">agg_all_unquoted</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">normalize_functions</span><span class="o">=</span><span class="s2">"lower"</span><span class="p">))</span>
|
||||
</span><span id="L-651"><a href="#L-651"><span class="linenos">651</span></a>
|
||||
</span><span id="L-652"><a href="#L-652"><span class="linenos">652</span></a> <span class="k">return</span> <span class="n">names</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -3209,6 +3221,31 @@ columns are removed from the create statement.</p>
|
|||
|
||||
|
||||
|
||||
</section>
|
||||
<section id="regexp_extract_sql">
|
||||
<input id="regexp_extract_sql-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">regexp_extract_sql</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span><span class="p">:</span> <span class="n"><a href="../generator.html#Generator">sqlglot.generator.Generator</a></span>,</span><span class="param"> <span class="n">expression</span><span class="p">:</span> <span class="n"><a href="../expressions.html#RegexpExtract">sqlglot.expressions.RegexpExtract</a></span></span><span class="return-annotation">) -> <span class="nb">str</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="regexp_extract_sql-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#regexp_extract_sql"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="regexp_extract_sql-624"><a href="#regexp_extract_sql-624"><span class="linenos">624</span></a><span class="k">def</span> <span class="nf">regexp_extract_sql</span><span class="p">(</span><span class="bp">self</span><span class="p">:</span> <span class="n">Generator</span><span class="p">,</span> <span class="n">expression</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">RegexpExtract</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="regexp_extract_sql-625"><a href="#regexp_extract_sql-625"><span class="linenos">625</span></a> <span class="n">bad_args</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">,</span> <span class="p">(</span><span class="s2">"position"</span><span class="p">,</span> <span class="s2">"occurrence"</span><span class="p">,</span> <span class="s2">"parameters"</span><span class="p">)))</span>
|
||||
</span><span id="regexp_extract_sql-626"><a href="#regexp_extract_sql-626"><span class="linenos">626</span></a> <span class="k">if</span> <span class="n">bad_args</span><span class="p">:</span>
|
||||
</span><span id="regexp_extract_sql-627"><a href="#regexp_extract_sql-627"><span class="linenos">627</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">unsupported</span><span class="p">(</span><span class="sa">f</span><span class="s2">"REGEXP_EXTRACT does not support the following arg(s): </span><span class="si">{</span><span class="n">bad_args</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="regexp_extract_sql-628"><a href="#regexp_extract_sql-628"><span class="linenos">628</span></a>
|
||||
</span><span id="regexp_extract_sql-629"><a href="#regexp_extract_sql-629"><span class="linenos">629</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">func</span><span class="p">(</span>
|
||||
</span><span id="regexp_extract_sql-630"><a href="#regexp_extract_sql-630"><span class="linenos">630</span></a> <span class="s2">"REGEXP_EXTRACT"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"group"</span><span class="p">)</span>
|
||||
</span><span id="regexp_extract_sql-631"><a href="#regexp_extract_sql-631"><span class="linenos">631</span></a> <span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
||||
|
||||
</section>
|
||||
<section id="pivot_column_names">
|
||||
<input id="pivot_column_names-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
|
|
@ -3221,26 +3258,26 @@ columns are removed from the create statement.</p>
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#pivot_column_names"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="pivot_column_names-625"><a href="#pivot_column_names-625"><span class="linenos">625</span></a><span class="k">def</span> <span class="nf">pivot_column_names</span><span class="p">(</span><span class="n">aggregations</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
||||
</span><span id="pivot_column_names-626"><a href="#pivot_column_names-626"><span class="linenos">626</span></a> <span class="n">names</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="pivot_column_names-627"><a href="#pivot_column_names-627"><span class="linenos">627</span></a> <span class="k">for</span> <span class="n">agg</span> <span class="ow">in</span> <span class="n">aggregations</span><span class="p">:</span>
|
||||
</span><span id="pivot_column_names-628"><a href="#pivot_column_names-628"><span class="linenos">628</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">agg</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="pivot_column_names-629"><a href="#pivot_column_names-629"><span class="linenos">629</span></a> <span class="n">names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">agg</span><span class="o">.</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="pivot_column_names-630"><a href="#pivot_column_names-630"><span class="linenos">630</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="pivot_column_names-631"><a href="#pivot_column_names-631"><span class="linenos">631</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="pivot_column_names-632"><a href="#pivot_column_names-632"><span class="linenos">632</span></a><span class="sd"> This case corresponds to aggregations without aliases being used as suffixes</span>
|
||||
</span><span id="pivot_column_names-633"><a href="#pivot_column_names-633"><span class="linenos">633</span></a><span class="sd"> (e.g. col_avg(foo)). We need to unquote identifiers because they're going to</span>
|
||||
</span><span id="pivot_column_names-634"><a href="#pivot_column_names-634"><span class="linenos">634</span></a><span class="sd"> be quoted in the base parser's `_parse_pivot` method, due to `to_identifier`.</span>
|
||||
</span><span id="pivot_column_names-635"><a href="#pivot_column_names-635"><span class="linenos">635</span></a><span class="sd"> Otherwise, we'd end up with `col_avg(`foo`)` (notice the double quotes).</span>
|
||||
</span><span id="pivot_column_names-636"><a href="#pivot_column_names-636"><span class="linenos">636</span></a><span class="sd"> """</span>
|
||||
</span><span id="pivot_column_names-637"><a href="#pivot_column_names-637"><span class="linenos">637</span></a> <span class="n">agg_all_unquoted</span> <span class="o">=</span> <span class="n">agg</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
|
||||
</span><span id="pivot_column_names-638"><a href="#pivot_column_names-638"><span class="linenos">638</span></a> <span class="k">lambda</span> <span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">quoted</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="pivot_column_names-639"><a href="#pivot_column_names-639"><span class="linenos">639</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span>
|
||||
</span><span id="pivot_column_names-640"><a href="#pivot_column_names-640"><span class="linenos">640</span></a> <span class="k">else</span> <span class="n">node</span>
|
||||
</span><span id="pivot_column_names-641"><a href="#pivot_column_names-641"><span class="linenos">641</span></a> <span class="p">)</span>
|
||||
</span><span id="pivot_column_names-642"><a href="#pivot_column_names-642"><span class="linenos">642</span></a> <span class="n">names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">agg_all_unquoted</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">normalize_functions</span><span class="o">=</span><span class="s2">"lower"</span><span class="p">))</span>
|
||||
</span><span id="pivot_column_names-643"><a href="#pivot_column_names-643"><span class="linenos">643</span></a>
|
||||
</span><span id="pivot_column_names-644"><a href="#pivot_column_names-644"><span class="linenos">644</span></a> <span class="k">return</span> <span class="n">names</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="pivot_column_names-634"><a href="#pivot_column_names-634"><span class="linenos">634</span></a><span class="k">def</span> <span class="nf">pivot_column_names</span><span class="p">(</span><span class="n">aggregations</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">],</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
||||
</span><span id="pivot_column_names-635"><a href="#pivot_column_names-635"><span class="linenos">635</span></a> <span class="n">names</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="pivot_column_names-636"><a href="#pivot_column_names-636"><span class="linenos">636</span></a> <span class="k">for</span> <span class="n">agg</span> <span class="ow">in</span> <span class="n">aggregations</span><span class="p">:</span>
|
||||
</span><span id="pivot_column_names-637"><a href="#pivot_column_names-637"><span class="linenos">637</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">agg</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="pivot_column_names-638"><a href="#pivot_column_names-638"><span class="linenos">638</span></a> <span class="n">names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">agg</span><span class="o">.</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="pivot_column_names-639"><a href="#pivot_column_names-639"><span class="linenos">639</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="pivot_column_names-640"><a href="#pivot_column_names-640"><span class="linenos">640</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="pivot_column_names-641"><a href="#pivot_column_names-641"><span class="linenos">641</span></a><span class="sd"> This case corresponds to aggregations without aliases being used as suffixes</span>
|
||||
</span><span id="pivot_column_names-642"><a href="#pivot_column_names-642"><span class="linenos">642</span></a><span class="sd"> (e.g. col_avg(foo)). We need to unquote identifiers because they're going to</span>
|
||||
</span><span id="pivot_column_names-643"><a href="#pivot_column_names-643"><span class="linenos">643</span></a><span class="sd"> be quoted in the base parser's `_parse_pivot` method, due to `to_identifier`.</span>
|
||||
</span><span id="pivot_column_names-644"><a href="#pivot_column_names-644"><span class="linenos">644</span></a><span class="sd"> Otherwise, we'd end up with `col_avg(`foo`)` (notice the double quotes).</span>
|
||||
</span><span id="pivot_column_names-645"><a href="#pivot_column_names-645"><span class="linenos">645</span></a><span class="sd"> """</span>
|
||||
</span><span id="pivot_column_names-646"><a href="#pivot_column_names-646"><span class="linenos">646</span></a> <span class="n">agg_all_unquoted</span> <span class="o">=</span> <span class="n">agg</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
|
||||
</span><span id="pivot_column_names-647"><a href="#pivot_column_names-647"><span class="linenos">647</span></a> <span class="k">lambda</span> <span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">quoted</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="pivot_column_names-648"><a href="#pivot_column_names-648"><span class="linenos">648</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span>
|
||||
</span><span id="pivot_column_names-649"><a href="#pivot_column_names-649"><span class="linenos">649</span></a> <span class="k">else</span> <span class="n">node</span>
|
||||
</span><span id="pivot_column_names-650"><a href="#pivot_column_names-650"><span class="linenos">650</span></a> <span class="p">)</span>
|
||||
</span><span id="pivot_column_names-651"><a href="#pivot_column_names-651"><span class="linenos">651</span></a> <span class="n">names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">agg_all_unquoted</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">normalize_functions</span><span class="o">=</span><span class="s2">"lower"</span><span class="p">))</span>
|
||||
</span><span id="pivot_column_names-652"><a href="#pivot_column_names-652"><span class="linenos">652</span></a>
|
||||
</span><span id="pivot_column_names-653"><a href="#pivot_column_names-653"><span class="linenos">653</span></a> <span class="k">return</span> <span class="n">names</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load diff
|
|
@ -216,7 +216,7 @@
|
|||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="c1"># Find the specific select clause that is the source of the column we want.</span>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="c1"># This can either be a specific, named select or a generic `*` clause.</span>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="n">select</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="p">(</span><span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">column_name</span><span class="p">),</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="p">(</span><span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">column_name</span><span class="p">),</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Star</span><span class="p">()</span> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">is_star</span> <span class="k">else</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="p">)</span>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a>
|
||||
|
|
@ -608,7 +608,7 @@
|
|||
</span><span id="lineage-105"><a href="#lineage-105"><span class="linenos">105</span></a> <span class="c1"># Find the specific select clause that is the source of the column we want.</span>
|
||||
</span><span id="lineage-106"><a href="#lineage-106"><span class="linenos">106</span></a> <span class="c1"># This can either be a specific, named select or a generic `*` clause.</span>
|
||||
</span><span id="lineage-107"><a href="#lineage-107"><span class="linenos">107</span></a> <span class="n">select</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span>
|
||||
</span><span id="lineage-108"><a href="#lineage-108"><span class="linenos">108</span></a> <span class="p">(</span><span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">column_name</span><span class="p">),</span>
|
||||
</span><span id="lineage-108"><a href="#lineage-108"><span class="linenos">108</span></a> <span class="p">(</span><span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">column_name</span><span class="p">),</span>
|
||||
</span><span id="lineage-109"><a href="#lineage-109"><span class="linenos">109</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Star</span><span class="p">()</span> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">is_star</span> <span class="k">else</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="lineage-110"><a href="#lineage-110"><span class="linenos">110</span></a> <span class="p">)</span>
|
||||
</span><span id="lineage-111"><a href="#lineage-111"><span class="linenos">111</span></a>
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -146,7 +146,7 @@
|
|||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="n">grouped_outputs</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="n">unique_outputs</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">output</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">unalias</span><span class="p">()</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="k">if</span> <span class="n">output</span> <span class="ow">in</span> <span class="n">grouped_expressions</span><span class="p">:</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="n">grouped_outputs</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">output</span><span class="p">)</span>
|
||||
|
|
@ -166,7 +166,7 @@
|
|||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a><span class="k">def</span> <span class="nf">_has_single_output_row</span><span class="p">(</span><span class="n">scope</span><span class="p">):</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">return</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="p">(</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="nb">all</span><span class="p">(</span><span class="nb">isinstance</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">unalias</span><span class="p">(),</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">)</span> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="nb">all</span><span class="p">(</span><span class="nb">isinstance</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">unalias</span><span class="p">(),</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">)</span> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="ow">or</span> <span class="n">_is_limit_1</span><span class="p">(</span><span class="n">scope</span><span class="p">)</span>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="ow">or</span> <span class="ow">not</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">)</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="p">)</span>
|
||||
|
|
|
|||
|
|
@ -171,7 +171,7 @@
|
|||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="n">taken</span><span class="p">[</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">scope</span>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="c1"># Try to maintain the selections</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">expressions</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">selects</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">expressions</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">selects</span> <span class="o">=</span> <span class="p">[</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="n">alias</span><span class="p">),</span> <span class="n">alias</span><span class="o">=</span><span class="n">e</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">expressions</span>
|
||||
|
|
|
|||
|
|
@ -70,22 +70,27 @@
|
|||
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a>
|
||||
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a> <span class="k">for</span> <span class="n">_</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
|
||||
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">schema</span><span class="o">.</span><span class="n">column_names</span><span class="p">(</span><span class="n">source</span><span class="p">):</span>
|
||||
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a>
|
||||
</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="s2">"Tables require an alias. Run qualify_tables optimization."</span><span class="p">)</span>
|
||||
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a> <span class="n">source</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos">22</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">"*"</span><span class="p">)</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos">23</span></a> <span class="o">.</span><span class="n">from_</span><span class="p">(</span>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos">24</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">source</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos">25</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos">26</span></a> <span class="p">)</span>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos">27</span></a> <span class="o">.</span><span class="n">subquery</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos">28</span></a> <span class="p">)</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos">29</span></a>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos">30</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
|
||||
</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a> <span class="ow">or</span> <span class="ow">not</span> <span class="n">schema</span><span class="o">.</span><span class="n">column_names</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
|
||||
</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">)</span>
|
||||
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
|
||||
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a> <span class="p">):</span>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos">22</span></a>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos">23</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos">24</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="s2">"Tables require an alias. Run qualify_tables optimization."</span><span class="p">)</span>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos">25</span></a>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos">26</span></a> <span class="n">source</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos">27</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">"*"</span><span class="p">)</span>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos">28</span></a> <span class="o">.</span><span class="n">from_</span><span class="p">(</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos">29</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">source</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos">30</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos">31</span></a> <span class="p">)</span>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos">32</span></a> <span class="o">.</span><span class="n">subquery</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos">33</span></a> <span class="p">)</span>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos">34</span></a>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos">35</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -109,22 +114,27 @@
|
|||
</span><span id="isolate_table_selects-13"><a href="#isolate_table_selects-13"><span class="linenos">13</span></a> <span class="k">continue</span>
|
||||
</span><span id="isolate_table_selects-14"><a href="#isolate_table_selects-14"><span class="linenos">14</span></a>
|
||||
</span><span id="isolate_table_selects-15"><a href="#isolate_table_selects-15"><span class="linenos">15</span></a> <span class="k">for</span> <span class="n">_</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
|
||||
</span><span id="isolate_table_selects-16"><a href="#isolate_table_selects-16"><span class="linenos">16</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">schema</span><span class="o">.</span><span class="n">column_names</span><span class="p">(</span><span class="n">source</span><span class="p">):</span>
|
||||
</span><span id="isolate_table_selects-17"><a href="#isolate_table_selects-17"><span class="linenos">17</span></a> <span class="k">continue</span>
|
||||
</span><span id="isolate_table_selects-18"><a href="#isolate_table_selects-18"><span class="linenos">18</span></a>
|
||||
</span><span id="isolate_table_selects-19"><a href="#isolate_table_selects-19"><span class="linenos">19</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="isolate_table_selects-20"><a href="#isolate_table_selects-20"><span class="linenos">20</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="s2">"Tables require an alias. Run qualify_tables optimization."</span><span class="p">)</span>
|
||||
</span><span id="isolate_table_selects-21"><a href="#isolate_table_selects-21"><span class="linenos">21</span></a>
|
||||
</span><span id="isolate_table_selects-22"><a href="#isolate_table_selects-22"><span class="linenos">22</span></a> <span class="n">source</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span>
|
||||
</span><span id="isolate_table_selects-23"><a href="#isolate_table_selects-23"><span class="linenos">23</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">"*"</span><span class="p">)</span>
|
||||
</span><span id="isolate_table_selects-24"><a href="#isolate_table_selects-24"><span class="linenos">24</span></a> <span class="o">.</span><span class="n">from_</span><span class="p">(</span>
|
||||
</span><span id="isolate_table_selects-25"><a href="#isolate_table_selects-25"><span class="linenos">25</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">source</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
|
||||
</span><span id="isolate_table_selects-26"><a href="#isolate_table_selects-26"><span class="linenos">26</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="isolate_table_selects-27"><a href="#isolate_table_selects-27"><span class="linenos">27</span></a> <span class="p">)</span>
|
||||
</span><span id="isolate_table_selects-28"><a href="#isolate_table_selects-28"><span class="linenos">28</span></a> <span class="o">.</span><span class="n">subquery</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="isolate_table_selects-29"><a href="#isolate_table_selects-29"><span class="linenos">29</span></a> <span class="p">)</span>
|
||||
</span><span id="isolate_table_selects-30"><a href="#isolate_table_selects-30"><span class="linenos">30</span></a>
|
||||
</span><span id="isolate_table_selects-31"><a href="#isolate_table_selects-31"><span class="linenos">31</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="isolate_table_selects-16"><a href="#isolate_table_selects-16"><span class="linenos">16</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="isolate_table_selects-17"><a href="#isolate_table_selects-17"><span class="linenos">17</span></a> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
|
||||
</span><span id="isolate_table_selects-18"><a href="#isolate_table_selects-18"><span class="linenos">18</span></a> <span class="ow">or</span> <span class="ow">not</span> <span class="n">schema</span><span class="o">.</span><span class="n">column_names</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
|
||||
</span><span id="isolate_table_selects-19"><a href="#isolate_table_selects-19"><span class="linenos">19</span></a> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">)</span>
|
||||
</span><span id="isolate_table_selects-20"><a href="#isolate_table_selects-20"><span class="linenos">20</span></a> <span class="ow">or</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
|
||||
</span><span id="isolate_table_selects-21"><a href="#isolate_table_selects-21"><span class="linenos">21</span></a> <span class="p">):</span>
|
||||
</span><span id="isolate_table_selects-22"><a href="#isolate_table_selects-22"><span class="linenos">22</span></a> <span class="k">continue</span>
|
||||
</span><span id="isolate_table_selects-23"><a href="#isolate_table_selects-23"><span class="linenos">23</span></a>
|
||||
</span><span id="isolate_table_selects-24"><a href="#isolate_table_selects-24"><span class="linenos">24</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="isolate_table_selects-25"><a href="#isolate_table_selects-25"><span class="linenos">25</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="s2">"Tables require an alias. Run qualify_tables optimization."</span><span class="p">)</span>
|
||||
</span><span id="isolate_table_selects-26"><a href="#isolate_table_selects-26"><span class="linenos">26</span></a>
|
||||
</span><span id="isolate_table_selects-27"><a href="#isolate_table_selects-27"><span class="linenos">27</span></a> <span class="n">source</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span>
|
||||
</span><span id="isolate_table_selects-28"><a href="#isolate_table_selects-28"><span class="linenos">28</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">"*"</span><span class="p">)</span>
|
||||
</span><span id="isolate_table_selects-29"><a href="#isolate_table_selects-29"><span class="linenos">29</span></a> <span class="o">.</span><span class="n">from_</span><span class="p">(</span>
|
||||
</span><span id="isolate_table_selects-30"><a href="#isolate_table_selects-30"><span class="linenos">30</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">source</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span>
|
||||
</span><span id="isolate_table_selects-31"><a href="#isolate_table_selects-31"><span class="linenos">31</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
|
||||
</span><span id="isolate_table_selects-32"><a href="#isolate_table_selects-32"><span class="linenos">32</span></a> <span class="p">)</span>
|
||||
</span><span id="isolate_table_selects-33"><a href="#isolate_table_selects-33"><span class="linenos">33</span></a> <span class="o">.</span><span class="n">subquery</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="isolate_table_selects-34"><a href="#isolate_table_selects-34"><span class="linenos">34</span></a> <span class="p">)</span>
|
||||
</span><span id="isolate_table_selects-35"><a href="#isolate_table_selects-35"><span class="linenos">35</span></a>
|
||||
</span><span id="isolate_table_selects-36"><a href="#isolate_table_selects-36"><span class="linenos">36</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -177,296 +177,297 @@
|
|||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="n">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">_merge_hints</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">clear_cache</span><span class="p">()</span>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a><span class="k">def</span> <span class="nf">_mergeable</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">):</span>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a><span class="sd"> Return True if `inner_select` can be merged into outer query.</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a><span class="sd"> outer_scope (Scope)</span>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a><span class="sd"> inner_scope (Scope)</span>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a><span class="sd"> leave_tables_isolated (bool)</span>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a><span class="sd"> from_or_join (exp.From|exp.Join)</span>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a><span class="sd"> bool: True if can be merged</span>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">inner_select</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="k">def</span> <span class="nf">_is_a_window_expression_in_unmergable_operation</span><span class="p">():</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">window_expressions</span> <span class="o">=</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">window_alias_names</span> <span class="o">=</span> <span class="p">{</span><span class="n">window</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="k">for</span> <span class="n">window</span> <span class="ow">in</span> <span class="n">window_expressions</span><span class="p">}</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="n">inner_select_name</span> <span class="o">=</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">unmergable_window_columns</span> <span class="o">=</span> <span class="p">[</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="n">column</span>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">columns</span>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Group</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Order</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="p">)</span>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="p">]</span>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="n">window_expressions_in_unmergable</span> <span class="o">=</span> <span class="p">[</span>
|
||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="n">column</span>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">unmergable_window_columns</span>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">inner_select_name</span> <span class="ow">and</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">window_alias_names</span>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="p">]</span>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="k">return</span> <span class="nb">any</span><span class="p">(</span><span class="n">window_expressions_in_unmergable</span><span class="p">)</span>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="k">def</span> <span class="nf">_outer_select_joins_on_inner_select_join</span><span class="p">():</span>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a><span class="sd"> All columns from the inner select in the ON clause must be from the first FROM table.</span>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a><span class="sd"> That is, this can be merged:</span>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a><span class="sd"> SELECT * FROM x JOIN (SELECT y.a AS a FROM y JOIN z) AS q ON x.a = q.a</span>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a><span class="sd"> ^^^ ^</span>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a><span class="sd"> But this can't:</span>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a><span class="sd"> SELECT * FROM x JOIN (SELECT z.a AS a FROM y JOIN z) AS q ON x.a = q.a</span>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a><span class="sd"> ^^^ ^</span>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="n">on</span> <span class="o">=</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">on</span><span class="p">:</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="n">selections</span> <span class="o">=</span> <span class="p">[</span><span class="n">c</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">on</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">alias</span><span class="p">]</span>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="n">inner_from</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">)</span>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">inner_from</span><span class="p">:</span>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="n">inner_from_table</span> <span class="o">=</span> <span class="n">inner_from</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="n">inner_projections</span> <span class="o">=</span> <span class="p">{</span><span class="n">s</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span> <span class="n">s</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">selects</span><span class="p">}</span>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="k">return</span> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="n">col</span><span class="o">.</span><span class="n">table</span> <span class="o">!=</span> <span class="n">inner_from_table</span>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="k">for</span> <span class="n">selection</span> <span class="ow">in</span> <span class="n">selections</span>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">inner_projections</span><span class="p">[</span><span class="n">selection</span><span class="p">]</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="p">)</span>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="k">return</span> <span class="p">(</span>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">inner_select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="n">UNMERGABLE_ARGS</span><span class="p">)</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">)</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">pivots</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span><span class="n">leave_tables_isolated</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span> <span class="o">></span> <span class="mi">1</span><span class="p">)</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">)</span>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">)</span>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="ow">and</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">side</span> <span class="ow">in</span> <span class="p">{</span><span class="s2">"FULL"</span><span class="p">,</span> <span class="s2">"LEFT"</span><span class="p">,</span> <span class="s2">"RIGHT"</span><span class="p">}</span>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="p">)</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">)</span>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="ow">and</span> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="n">j</span><span class="o">.</span><span class="n">side</span> <span class="ow">in</span> <span class="p">{</span><span class="s2">"FULL"</span><span class="p">,</span> <span class="s2">"RIGHT"</span><span class="p">}</span> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="p">)</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="p">)</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">_outer_select_joins_on_inner_select_join</span><span class="p">()</span>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">_is_a_window_expression_in_unmergable_operation</span><span class="p">()</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="p">)</span>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a><span class="k">def</span> <span class="nf">_mergeable</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">leave_tables_isolated</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">):</span>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a><span class="sd"> Return True if `inner_select` can be merged into outer query.</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a><span class="sd"> outer_scope (Scope)</span>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a><span class="sd"> inner_scope (Scope)</span>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a><span class="sd"> leave_tables_isolated (bool)</span>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a><span class="sd"> from_or_join (exp.From|exp.Join)</span>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a><span class="sd"> bool: True if can be merged</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="n">inner_select</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">def</span> <span class="nf">_is_a_window_expression_in_unmergable_operation</span><span class="p">():</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">window_expressions</span> <span class="o">=</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="n">window_alias_names</span> <span class="o">=</span> <span class="p">{</span><span class="n">window</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="k">for</span> <span class="n">window</span> <span class="ow">in</span> <span class="n">window_expressions</span><span class="p">}</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">inner_select_name</span> <span class="o">=</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="n">unmergable_window_columns</span> <span class="o">=</span> <span class="p">[</span>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="n">column</span>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">columns</span>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Group</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Order</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Having</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="p">)</span>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="p">]</span>
|
||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="n">window_expressions_in_unmergable</span> <span class="o">=</span> <span class="p">[</span>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">column</span>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">unmergable_window_columns</span>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">inner_select_name</span> <span class="ow">and</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">window_alias_names</span>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="p">]</span>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="k">return</span> <span class="nb">any</span><span class="p">(</span><span class="n">window_expressions_in_unmergable</span><span class="p">)</span>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="k">def</span> <span class="nf">_outer_select_joins_on_inner_select_join</span><span class="p">():</span>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a><span class="sd"> All columns from the inner select in the ON clause must be from the first FROM table.</span>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a><span class="sd"> That is, this can be merged:</span>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a><span class="sd"> SELECT * FROM x JOIN (SELECT y.a AS a FROM y JOIN z) AS q ON x.a = q.a</span>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a><span class="sd"> ^^^ ^</span>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a><span class="sd"> But this can't:</span>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a><span class="sd"> SELECT * FROM x JOIN (SELECT z.a AS a FROM y JOIN z) AS q ON x.a = q.a</span>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a><span class="sd"> ^^^ ^</span>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="n">on</span> <span class="o">=</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">on</span><span class="p">:</span>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="n">selections</span> <span class="o">=</span> <span class="p">[</span><span class="n">c</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">on</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">alias</span><span class="p">]</span>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="n">inner_from</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">)</span>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">inner_from</span><span class="p">:</span>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="n">inner_from_table</span> <span class="o">=</span> <span class="n">inner_from</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="n">inner_projections</span> <span class="o">=</span> <span class="p">{</span><span class="n">s</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span> <span class="n">s</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">}</span>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="k">return</span> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="n">col</span><span class="o">.</span><span class="n">table</span> <span class="o">!=</span> <span class="n">inner_from_table</span>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="k">for</span> <span class="n">selection</span> <span class="ow">in</span> <span class="n">selections</span>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">inner_projections</span><span class="p">[</span><span class="n">selection</span><span class="p">]</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="p">)</span>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="k">return</span> <span class="p">(</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">inner_select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="n">UNMERGABLE_ARGS</span><span class="p">)</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">)</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">pivots</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">e</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span><span class="n">leave_tables_isolated</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span> <span class="o">></span> <span class="mi">1</span><span class="p">)</span>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">)</span>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">)</span>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="ow">and</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">side</span> <span class="ow">in</span> <span class="p">{</span><span class="s2">"FULL"</span><span class="p">,</span> <span class="s2">"LEFT"</span><span class="p">,</span> <span class="s2">"RIGHT"</span><span class="p">}</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="p">)</span>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="p">(</span>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="ow">and</span> <span class="n">inner_select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">)</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="ow">and</span> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="n">j</span><span class="o">.</span><span class="n">side</span> <span class="ow">in</span> <span class="p">{</span><span class="s2">"FULL"</span><span class="p">,</span> <span class="s2">"RIGHT"</span><span class="p">}</span> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="p">)</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="p">)</span>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">_outer_select_joins_on_inner_select_join</span><span class="p">()</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">_is_a_window_expression_in_unmergable_operation</span><span class="p">()</span>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="p">)</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a><span class="k">def</span> <span class="nf">_rename_inner_sources</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a><span class="sd"> Renames any sources in the inner query that conflict with names in the outer query.</span>
|
||||
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a>
|
||||
</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a><span class="sd"> alias (str)</span>
|
||||
</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a> <span class="n">taken</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span>
|
||||
</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="n">conflicts</span> <span class="o">=</span> <span class="n">taken</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">inner_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">))</span>
|
||||
</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a> <span class="n">conflicts</span> <span class="o">-=</span> <span class="p">{</span><span class="n">alias</span><span class="p">}</span>
|
||||
</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a>
|
||||
</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a> <span class="k">for</span> <span class="n">conflict</span> <span class="ow">in</span> <span class="n">conflicts</span><span class="p">:</span>
|
||||
</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a> <span class="n">new_name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="p">,</span> <span class="n">conflict</span><span class="p">)</span>
|
||||
</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a>
|
||||
</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a> <span class="n">source</span><span class="p">,</span> <span class="n">_</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">conflict</span><span class="p">]</span>
|
||||
</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a> <span class="n">new_alias</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_name</span><span class="p">)</span>
|
||||
</span><span id="L-219"><a href="#L-219"><span class="linenos">219</span></a>
|
||||
</span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
|
||||
</span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">new_alias</span><span class="p">))</span>
|
||||
</span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-223"><a href="#L-223"><span class="linenos">223</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">new_alias</span><span class="p">)</span>
|
||||
</span><span id="L-224"><a href="#L-224"><span class="linenos">224</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-225"><a href="#L-225"><span class="linenos">225</span></a> <span class="n">source</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">new_alias</span><span class="p">))</span>
|
||||
</span><span id="L-226"><a href="#L-226"><span class="linenos">226</span></a>
|
||||
</span><span id="L-227"><a href="#L-227"><span class="linenos">227</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">source_columns</span><span class="p">(</span><span class="n">conflict</span><span class="p">):</span>
|
||||
</span><span id="L-228"><a href="#L-228"><span class="linenos">228</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"table"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_name</span><span class="p">))</span>
|
||||
</span><span id="L-229"><a href="#L-229"><span class="linenos">229</span></a>
|
||||
</span><span id="L-230"><a href="#L-230"><span class="linenos">230</span></a> <span class="n">inner_scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="n">conflict</span><span class="p">,</span> <span class="n">new_name</span><span class="p">)</span>
|
||||
</span><span id="L-231"><a href="#L-231"><span class="linenos">231</span></a>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a>
|
||||
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a><span class="k">def</span> <span class="nf">_rename_inner_sources</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a><span class="sd"> Renames any sources in the inner query that conflict with names in the outer query.</span>
|
||||
</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a>
|
||||
</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a><span class="sd"> alias (str)</span>
|
||||
</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="n">taken</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span>
|
||||
</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a> <span class="n">conflicts</span> <span class="o">=</span> <span class="n">taken</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">inner_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">))</span>
|
||||
</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="n">conflicts</span> <span class="o">-=</span> <span class="p">{</span><span class="n">alias</span><span class="p">}</span>
|
||||
</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a>
|
||||
</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a> <span class="k">for</span> <span class="n">conflict</span> <span class="ow">in</span> <span class="n">conflicts</span><span class="p">:</span>
|
||||
</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a> <span class="n">new_name</span> <span class="o">=</span> <span class="n">find_new_name</span><span class="p">(</span><span class="n">taken</span><span class="p">,</span> <span class="n">conflict</span><span class="p">)</span>
|
||||
</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a>
|
||||
</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a> <span class="n">source</span><span class="p">,</span> <span class="n">_</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">conflict</span><span class="p">]</span>
|
||||
</span><span id="L-219"><a href="#L-219"><span class="linenos">219</span></a> <span class="n">new_alias</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_name</span><span class="p">)</span>
|
||||
</span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a>
|
||||
</span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
|
||||
</span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">new_alias</span><span class="p">))</span>
|
||||
</span><span id="L-223"><a href="#L-223"><span class="linenos">223</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-224"><a href="#L-224"><span class="linenos">224</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">new_alias</span><span class="p">)</span>
|
||||
</span><span id="L-225"><a href="#L-225"><span class="linenos">225</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-226"><a href="#L-226"><span class="linenos">226</span></a> <span class="n">source</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">alias_</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">new_alias</span><span class="p">))</span>
|
||||
</span><span id="L-227"><a href="#L-227"><span class="linenos">227</span></a>
|
||||
</span><span id="L-228"><a href="#L-228"><span class="linenos">228</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">source_columns</span><span class="p">(</span><span class="n">conflict</span><span class="p">):</span>
|
||||
</span><span id="L-229"><a href="#L-229"><span class="linenos">229</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"table"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_name</span><span class="p">))</span>
|
||||
</span><span id="L-230"><a href="#L-230"><span class="linenos">230</span></a>
|
||||
</span><span id="L-231"><a href="#L-231"><span class="linenos">231</span></a> <span class="n">inner_scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="n">conflict</span><span class="p">,</span> <span class="n">new_name</span><span class="p">)</span>
|
||||
</span><span id="L-232"><a href="#L-232"><span class="linenos">232</span></a>
|
||||
</span><span id="L-233"><a href="#L-233"><span class="linenos">233</span></a><span class="k">def</span> <span class="nf">_merge_from</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">node_to_replace</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-234"><a href="#L-234"><span class="linenos">234</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-235"><a href="#L-235"><span class="linenos">235</span></a><span class="sd"> Merge FROM clause of inner query into outer query.</span>
|
||||
</span><span id="L-236"><a href="#L-236"><span class="linenos">236</span></a>
|
||||
</span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a><span class="sd"> node_to_replace (exp.Subquery|exp.Table)</span>
|
||||
</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a><span class="sd"> alias (str)</span>
|
||||
</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a> <span class="n">new_subquery</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"from"</span><span class="p">]</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="n">node_to_replace</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">new_subquery</span><span class="p">)</span>
|
||||
</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a> <span class="k">for</span> <span class="n">join_hint</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">join_hints</span><span class="p">:</span>
|
||||
</span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">join_hint</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
|
||||
</span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">:</span>
|
||||
</span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a> <span class="k">if</span> <span class="n">table</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">node_to_replace</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span>
|
||||
</span><span id="L-249"><a href="#L-249"><span class="linenos">249</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"this"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_subquery</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">))</span>
|
||||
</span><span id="L-250"><a href="#L-250"><span class="linenos">250</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">remove_source</span><span class="p">(</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="L-251"><a href="#L-251"><span class="linenos">251</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">add_source</span><span class="p">(</span>
|
||||
</span><span id="L-252"><a href="#L-252"><span class="linenos">252</span></a> <span class="n">new_subquery</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">new_subquery</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">]</span>
|
||||
</span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a> <span class="p">)</span>
|
||||
</span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a>
|
||||
</span><span id="L-233"><a href="#L-233"><span class="linenos">233</span></a>
|
||||
</span><span id="L-234"><a href="#L-234"><span class="linenos">234</span></a><span class="k">def</span> <span class="nf">_merge_from</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">node_to_replace</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-235"><a href="#L-235"><span class="linenos">235</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-236"><a href="#L-236"><span class="linenos">236</span></a><span class="sd"> Merge FROM clause of inner query into outer query.</span>
|
||||
</span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a>
|
||||
</span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a><span class="sd"> node_to_replace (exp.Subquery|exp.Table)</span>
|
||||
</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a><span class="sd"> alias (str)</span>
|
||||
</span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="n">new_subquery</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"from"</span><span class="p">]</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a> <span class="n">node_to_replace</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">new_subquery</span><span class="p">)</span>
|
||||
</span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a> <span class="k">for</span> <span class="n">join_hint</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">join_hints</span><span class="p">:</span>
|
||||
</span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">join_hint</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">)</span>
|
||||
</span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">:</span>
|
||||
</span><span id="L-249"><a href="#L-249"><span class="linenos">249</span></a> <span class="k">if</span> <span class="n">table</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="o">==</span> <span class="n">node_to_replace</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span>
|
||||
</span><span id="L-250"><a href="#L-250"><span class="linenos">250</span></a> <span class="n">table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"this"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">new_subquery</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">))</span>
|
||||
</span><span id="L-251"><a href="#L-251"><span class="linenos">251</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">remove_source</span><span class="p">(</span><span class="n">alias</span><span class="p">)</span>
|
||||
</span><span id="L-252"><a href="#L-252"><span class="linenos">252</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">add_source</span><span class="p">(</span>
|
||||
</span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a> <span class="n">new_subquery</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">new_subquery</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">]</span>
|
||||
</span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a> <span class="p">)</span>
|
||||
</span><span id="L-255"><a href="#L-255"><span class="linenos">255</span></a>
|
||||
</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a><span class="k">def</span> <span class="nf">_merge_joins</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">):</span>
|
||||
</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a><span class="sd"> Merge JOIN clauses of inner query into outer query.</span>
|
||||
</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a>
|
||||
</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a><span class="sd"> from_or_join (exp.From|exp.Join)</span>
|
||||
</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a>
|
||||
</span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a> <span class="n">new_joins</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a>
|
||||
</span><span id="L-268"><a href="#L-268"><span class="linenos">268</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="p">:</span>
|
||||
</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a> <span class="n">new_joins</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">add_source</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">])</span>
|
||||
</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a>
|
||||
</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a> <span class="k">if</span> <span class="n">new_joins</span><span class="p">:</span>
|
||||
</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a> <span class="n">outer_joins</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a>
|
||||
</span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a> <span class="c1"># Maintain the join order</span>
|
||||
</span><span id="L-277"><a href="#L-277"><span class="linenos">277</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">):</span>
|
||||
</span><span id="L-278"><a href="#L-278"><span class="linenos">278</span></a> <span class="n">position</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
</span><span id="L-279"><a href="#L-279"><span class="linenos">279</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-280"><a href="#L-280"><span class="linenos">280</span></a> <span class="n">position</span> <span class="o">=</span> <span class="n">outer_joins</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
|
||||
</span><span id="L-281"><a href="#L-281"><span class="linenos">281</span></a> <span class="n">outer_joins</span><span class="p">[</span><span class="n">position</span><span class="p">:</span><span class="n">position</span><span class="p">]</span> <span class="o">=</span> <span class="n">new_joins</span>
|
||||
</span><span id="L-282"><a href="#L-282"><span class="linenos">282</span></a>
|
||||
</span><span id="L-283"><a href="#L-283"><span class="linenos">283</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">outer_joins</span><span class="p">)</span>
|
||||
</span><span id="L-284"><a href="#L-284"><span class="linenos">284</span></a>
|
||||
</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a>
|
||||
</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a><span class="k">def</span> <span class="nf">_merge_joins</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">):</span>
|
||||
</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a><span class="sd"> Merge JOIN clauses of inner query into outer query.</span>
|
||||
</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a>
|
||||
</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a><span class="sd"> from_or_join (exp.From|exp.Join)</span>
|
||||
</span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a>
|
||||
</span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a> <span class="n">new_joins</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-268"><a href="#L-268"><span class="linenos">268</span></a>
|
||||
</span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="p">:</span>
|
||||
</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a> <span class="n">new_joins</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">add_source</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">])</span>
|
||||
</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a>
|
||||
</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a> <span class="k">if</span> <span class="n">new_joins</span><span class="p">:</span>
|
||||
</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a> <span class="n">outer_joins</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a>
|
||||
</span><span id="L-277"><a href="#L-277"><span class="linenos">277</span></a> <span class="c1"># Maintain the join order</span>
|
||||
</span><span id="L-278"><a href="#L-278"><span class="linenos">278</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">):</span>
|
||||
</span><span id="L-279"><a href="#L-279"><span class="linenos">279</span></a> <span class="n">position</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
</span><span id="L-280"><a href="#L-280"><span class="linenos">280</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-281"><a href="#L-281"><span class="linenos">281</span></a> <span class="n">position</span> <span class="o">=</span> <span class="n">outer_joins</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
|
||||
</span><span id="L-282"><a href="#L-282"><span class="linenos">282</span></a> <span class="n">outer_joins</span><span class="p">[</span><span class="n">position</span><span class="p">:</span><span class="n">position</span><span class="p">]</span> <span class="o">=</span> <span class="n">new_joins</span>
|
||||
</span><span id="L-283"><a href="#L-283"><span class="linenos">283</span></a>
|
||||
</span><span id="L-284"><a href="#L-284"><span class="linenos">284</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">outer_joins</span><span class="p">)</span>
|
||||
</span><span id="L-285"><a href="#L-285"><span class="linenos">285</span></a>
|
||||
</span><span id="L-286"><a href="#L-286"><span class="linenos">286</span></a><span class="k">def</span> <span class="nf">_merge_expressions</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-287"><a href="#L-287"><span class="linenos">287</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-288"><a href="#L-288"><span class="linenos">288</span></a><span class="sd"> Merge projections of inner query into outer query.</span>
|
||||
</span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a>
|
||||
</span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a><span class="sd"> alias (str)</span>
|
||||
</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a> <span class="c1"># Collect all columns that reference the alias of the inner query</span>
|
||||
</span><span id="L-296"><a href="#L-296"><span class="linenos">296</span></a> <span class="n">outer_columns</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">list</span><span class="p">)</span>
|
||||
</span><span id="L-297"><a href="#L-297"><span class="linenos">297</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-298"><a href="#L-298"><span class="linenos">298</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-299"><a href="#L-299"><span class="linenos">299</span></a> <span class="n">outer_columns</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">column</span><span class="p">)</span>
|
||||
</span><span id="L-300"><a href="#L-300"><span class="linenos">300</span></a>
|
||||
</span><span id="L-301"><a href="#L-301"><span class="linenos">301</span></a> <span class="c1"># Replace columns with the projection expression in the inner query</span>
|
||||
</span><span id="L-302"><a href="#L-302"><span class="linenos">302</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-303"><a href="#L-303"><span class="linenos">303</span></a> <span class="n">projection_name</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-304"><a href="#L-304"><span class="linenos">304</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">projection_name</span><span class="p">:</span>
|
||||
</span><span id="L-305"><a href="#L-305"><span class="linenos">305</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-306"><a href="#L-306"><span class="linenos">306</span></a> <span class="n">columns_to_replace</span> <span class="o">=</span> <span class="n">outer_columns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">projection_name</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-307"><a href="#L-307"><span class="linenos">307</span></a>
|
||||
</span><span id="L-308"><a href="#L-308"><span class="linenos">308</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">unalias</span><span class="p">()</span>
|
||||
</span><span id="L-309"><a href="#L-309"><span class="linenos">309</span></a> <span class="n">must_wrap_expression</span> <span class="o">=</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">SAFE_TO_REPLACE_UNWRAPPED</span><span class="p">)</span>
|
||||
</span><span id="L-310"><a href="#L-310"><span class="linenos">310</span></a>
|
||||
</span><span id="L-311"><a href="#L-311"><span class="linenos">311</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns_to_replace</span><span class="p">:</span>
|
||||
</span><span id="L-312"><a href="#L-312"><span class="linenos">312</span></a> <span class="c1"># Ensures we don't alter the intended operator precedence if there's additional</span>
|
||||
</span><span id="L-313"><a href="#L-313"><span class="linenos">313</span></a> <span class="c1"># context surrounding the outer expression (i.e. it's not a simple projection).</span>
|
||||
</span><span id="L-314"><a href="#L-314"><span class="linenos">314</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Unary</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">))</span> <span class="ow">and</span> <span class="n">must_wrap_expression</span><span class="p">:</span>
|
||||
</span><span id="L-315"><a href="#L-315"><span class="linenos">315</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">paren</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-316"><a href="#L-316"><span class="linenos">316</span></a>
|
||||
</span><span id="L-317"><a href="#L-317"><span class="linenos">317</span></a> <span class="n">column</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">())</span>
|
||||
</span><span id="L-318"><a href="#L-318"><span class="linenos">318</span></a>
|
||||
</span><span id="L-286"><a href="#L-286"><span class="linenos">286</span></a>
|
||||
</span><span id="L-287"><a href="#L-287"><span class="linenos">287</span></a><span class="k">def</span> <span class="nf">_merge_expressions</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-288"><a href="#L-288"><span class="linenos">288</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a><span class="sd"> Merge projections of inner query into outer query.</span>
|
||||
</span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a>
|
||||
</span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a><span class="sd"> alias (str)</span>
|
||||
</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-296"><a href="#L-296"><span class="linenos">296</span></a> <span class="c1"># Collect all columns that reference the alias of the inner query</span>
|
||||
</span><span id="L-297"><a href="#L-297"><span class="linenos">297</span></a> <span class="n">outer_columns</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">list</span><span class="p">)</span>
|
||||
</span><span id="L-298"><a href="#L-298"><span class="linenos">298</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-299"><a href="#L-299"><span class="linenos">299</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-300"><a href="#L-300"><span class="linenos">300</span></a> <span class="n">outer_columns</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">column</span><span class="p">)</span>
|
||||
</span><span id="L-301"><a href="#L-301"><span class="linenos">301</span></a>
|
||||
</span><span id="L-302"><a href="#L-302"><span class="linenos">302</span></a> <span class="c1"># Replace columns with the projection expression in the inner query</span>
|
||||
</span><span id="L-303"><a href="#L-303"><span class="linenos">303</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-304"><a href="#L-304"><span class="linenos">304</span></a> <span class="n">projection_name</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-305"><a href="#L-305"><span class="linenos">305</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">projection_name</span><span class="p">:</span>
|
||||
</span><span id="L-306"><a href="#L-306"><span class="linenos">306</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-307"><a href="#L-307"><span class="linenos">307</span></a> <span class="n">columns_to_replace</span> <span class="o">=</span> <span class="n">outer_columns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">projection_name</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-308"><a href="#L-308"><span class="linenos">308</span></a>
|
||||
</span><span id="L-309"><a href="#L-309"><span class="linenos">309</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">unalias</span><span class="p">()</span>
|
||||
</span><span id="L-310"><a href="#L-310"><span class="linenos">310</span></a> <span class="n">must_wrap_expression</span> <span class="o">=</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">SAFE_TO_REPLACE_UNWRAPPED</span><span class="p">)</span>
|
||||
</span><span id="L-311"><a href="#L-311"><span class="linenos">311</span></a>
|
||||
</span><span id="L-312"><a href="#L-312"><span class="linenos">312</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns_to_replace</span><span class="p">:</span>
|
||||
</span><span id="L-313"><a href="#L-313"><span class="linenos">313</span></a> <span class="c1"># Ensures we don't alter the intended operator precedence if there's additional</span>
|
||||
</span><span id="L-314"><a href="#L-314"><span class="linenos">314</span></a> <span class="c1"># context surrounding the outer expression (i.e. it's not a simple projection).</span>
|
||||
</span><span id="L-315"><a href="#L-315"><span class="linenos">315</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Unary</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Binary</span><span class="p">))</span> <span class="ow">and</span> <span class="n">must_wrap_expression</span><span class="p">:</span>
|
||||
</span><span id="L-316"><a href="#L-316"><span class="linenos">316</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">paren</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-317"><a href="#L-317"><span class="linenos">317</span></a>
|
||||
</span><span id="L-318"><a href="#L-318"><span class="linenos">318</span></a> <span class="n">column</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">copy</span><span class="p">())</span>
|
||||
</span><span id="L-319"><a href="#L-319"><span class="linenos">319</span></a>
|
||||
</span><span id="L-320"><a href="#L-320"><span class="linenos">320</span></a><span class="k">def</span> <span class="nf">_merge_where</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">):</span>
|
||||
</span><span id="L-321"><a href="#L-321"><span class="linenos">321</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-322"><a href="#L-322"><span class="linenos">322</span></a><span class="sd"> Merge WHERE clause of inner query into outer query.</span>
|
||||
</span><span id="L-323"><a href="#L-323"><span class="linenos">323</span></a>
|
||||
</span><span id="L-324"><a href="#L-324"><span class="linenos">324</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-325"><a href="#L-325"><span class="linenos">325</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-326"><a href="#L-326"><span class="linenos">326</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-327"><a href="#L-327"><span class="linenos">327</span></a><span class="sd"> from_or_join (exp.From|exp.Join)</span>
|
||||
</span><span id="L-328"><a href="#L-328"><span class="linenos">328</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-329"><a href="#L-329"><span class="linenos">329</span></a> <span class="n">where</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">)</span>
|
||||
</span><span id="L-330"><a href="#L-330"><span class="linenos">330</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">where</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">:</span>
|
||||
</span><span id="L-331"><a href="#L-331"><span class="linenos">331</span></a> <span class="k">return</span>
|
||||
</span><span id="L-332"><a href="#L-332"><span class="linenos">332</span></a>
|
||||
</span><span id="L-333"><a href="#L-333"><span class="linenos">333</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-334"><a href="#L-334"><span class="linenos">334</span></a>
|
||||
</span><span id="L-335"><a href="#L-335"><span class="linenos">335</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-336"><a href="#L-336"><span class="linenos">336</span></a> <span class="c1"># Merge predicates from an outer join to the ON clause</span>
|
||||
</span><span id="L-337"><a href="#L-337"><span class="linenos">337</span></a> <span class="c1"># if it only has columns that are already joined</span>
|
||||
</span><span id="L-338"><a href="#L-338"><span class="linenos">338</span></a> <span class="n">from_</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">)</span>
|
||||
</span><span id="L-339"><a href="#L-339"><span class="linenos">339</span></a> <span class="n">sources</span> <span class="o">=</span> <span class="p">{</span><span class="n">from_</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">}</span> <span class="k">if</span> <span class="n">from_</span> <span class="k">else</span> <span class="p">{}</span>
|
||||
</span><span id="L-340"><a href="#L-340"><span class="linenos">340</span></a>
|
||||
</span><span id="L-341"><a href="#L-341"><span class="linenos">341</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"joins"</span><span class="p">]:</span>
|
||||
</span><span id="L-342"><a href="#L-342"><span class="linenos">342</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-343"><a href="#L-343"><span class="linenos">343</span></a> <span class="n">sources</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
|
||||
</span><span id="L-344"><a href="#L-344"><span class="linenos">344</span></a> <span class="k">if</span> <span class="n">source</span> <span class="o">==</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span>
|
||||
</span><span id="L-345"><a href="#L-345"><span class="linenos">345</span></a> <span class="k">break</span>
|
||||
</span><span id="L-346"><a href="#L-346"><span class="linenos">346</span></a>
|
||||
</span><span id="L-347"><a href="#L-347"><span class="linenos">347</span></a> <span class="k">if</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="o"><=</span> <span class="n">sources</span><span class="p">:</span>
|
||||
</span><span id="L-348"><a href="#L-348"><span class="linenos">348</span></a> <span class="n">from_or_join</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-349"><a href="#L-349"><span class="linenos">349</span></a> <span class="n">from_or_join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"on"</span><span class="p">,</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">))</span>
|
||||
</span><span id="L-350"><a href="#L-350"><span class="linenos">350</span></a> <span class="k">return</span>
|
||||
</span><span id="L-351"><a href="#L-351"><span class="linenos">351</span></a>
|
||||
</span><span id="L-352"><a href="#L-352"><span class="linenos">352</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-353"><a href="#L-353"><span class="linenos">353</span></a>
|
||||
</span><span id="L-320"><a href="#L-320"><span class="linenos">320</span></a>
|
||||
</span><span id="L-321"><a href="#L-321"><span class="linenos">321</span></a><span class="k">def</span> <span class="nf">_merge_where</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">):</span>
|
||||
</span><span id="L-322"><a href="#L-322"><span class="linenos">322</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-323"><a href="#L-323"><span class="linenos">323</span></a><span class="sd"> Merge WHERE clause of inner query into outer query.</span>
|
||||
</span><span id="L-324"><a href="#L-324"><span class="linenos">324</span></a>
|
||||
</span><span id="L-325"><a href="#L-325"><span class="linenos">325</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-326"><a href="#L-326"><span class="linenos">326</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-327"><a href="#L-327"><span class="linenos">327</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-328"><a href="#L-328"><span class="linenos">328</span></a><span class="sd"> from_or_join (exp.From|exp.Join)</span>
|
||||
</span><span id="L-329"><a href="#L-329"><span class="linenos">329</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-330"><a href="#L-330"><span class="linenos">330</span></a> <span class="n">where</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">)</span>
|
||||
</span><span id="L-331"><a href="#L-331"><span class="linenos">331</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">where</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">:</span>
|
||||
</span><span id="L-332"><a href="#L-332"><span class="linenos">332</span></a> <span class="k">return</span>
|
||||
</span><span id="L-333"><a href="#L-333"><span class="linenos">333</span></a>
|
||||
</span><span id="L-334"><a href="#L-334"><span class="linenos">334</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-335"><a href="#L-335"><span class="linenos">335</span></a>
|
||||
</span><span id="L-336"><a href="#L-336"><span class="linenos">336</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-337"><a href="#L-337"><span class="linenos">337</span></a> <span class="c1"># Merge predicates from an outer join to the ON clause</span>
|
||||
</span><span id="L-338"><a href="#L-338"><span class="linenos">338</span></a> <span class="c1"># if it only has columns that are already joined</span>
|
||||
</span><span id="L-339"><a href="#L-339"><span class="linenos">339</span></a> <span class="n">from_</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">)</span>
|
||||
</span><span id="L-340"><a href="#L-340"><span class="linenos">340</span></a> <span class="n">sources</span> <span class="o">=</span> <span class="p">{</span><span class="n">from_</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">}</span> <span class="k">if</span> <span class="n">from_</span> <span class="k">else</span> <span class="p">{}</span>
|
||||
</span><span id="L-341"><a href="#L-341"><span class="linenos">341</span></a>
|
||||
</span><span id="L-342"><a href="#L-342"><span class="linenos">342</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"joins"</span><span class="p">]:</span>
|
||||
</span><span id="L-343"><a href="#L-343"><span class="linenos">343</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-344"><a href="#L-344"><span class="linenos">344</span></a> <span class="n">sources</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
|
||||
</span><span id="L-345"><a href="#L-345"><span class="linenos">345</span></a> <span class="k">if</span> <span class="n">source</span> <span class="o">==</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span>
|
||||
</span><span id="L-346"><a href="#L-346"><span class="linenos">346</span></a> <span class="k">break</span>
|
||||
</span><span id="L-347"><a href="#L-347"><span class="linenos">347</span></a>
|
||||
</span><span id="L-348"><a href="#L-348"><span class="linenos">348</span></a> <span class="k">if</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="o"><=</span> <span class="n">sources</span><span class="p">:</span>
|
||||
</span><span id="L-349"><a href="#L-349"><span class="linenos">349</span></a> <span class="n">from_or_join</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-350"><a href="#L-350"><span class="linenos">350</span></a> <span class="n">from_or_join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"on"</span><span class="p">,</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">))</span>
|
||||
</span><span id="L-351"><a href="#L-351"><span class="linenos">351</span></a> <span class="k">return</span>
|
||||
</span><span id="L-352"><a href="#L-352"><span class="linenos">352</span></a>
|
||||
</span><span id="L-353"><a href="#L-353"><span class="linenos">353</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-354"><a href="#L-354"><span class="linenos">354</span></a>
|
||||
</span><span id="L-355"><a href="#L-355"><span class="linenos">355</span></a><span class="k">def</span> <span class="nf">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-356"><a href="#L-356"><span class="linenos">356</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-357"><a href="#L-357"><span class="linenos">357</span></a><span class="sd"> Merge ORDER clause of inner query into outer query.</span>
|
||||
</span><span id="L-358"><a href="#L-358"><span class="linenos">358</span></a>
|
||||
</span><span id="L-359"><a href="#L-359"><span class="linenos">359</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-360"><a href="#L-360"><span class="linenos">360</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-361"><a href="#L-361"><span class="linenos">361</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-362"><a href="#L-362"><span class="linenos">362</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-363"><a href="#L-363"><span class="linenos">363</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-364"><a href="#L-364"><span class="linenos">364</span></a> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-365"><a href="#L-365"><span class="linenos">365</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="p">[</span><span class="s2">"group"</span><span class="p">,</span> <span class="s2">"distinct"</span><span class="p">,</span> <span class="s2">"having"</span><span class="p">,</span> <span class="s2">"order"</span><span class="p">]</span>
|
||||
</span><span id="L-366"><a href="#L-366"><span class="linenos">366</span></a> <span class="p">)</span>
|
||||
</span><span id="L-367"><a href="#L-367"><span class="linenos">367</span></a> <span class="ow">or</span> <span class="nb">len</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span> <span class="o">!=</span> <span class="mi">1</span>
|
||||
</span><span id="L-368"><a href="#L-368"><span class="linenos">368</span></a> <span class="ow">or</span> <span class="nb">any</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">)</span> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span>
|
||||
</span><span id="L-369"><a href="#L-369"><span class="linenos">369</span></a> <span class="p">):</span>
|
||||
</span><span id="L-370"><a href="#L-370"><span class="linenos">370</span></a> <span class="k">return</span>
|
||||
</span><span id="L-371"><a href="#L-371"><span class="linenos">371</span></a>
|
||||
</span><span id="L-372"><a href="#L-372"><span class="linenos">372</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"order"</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"order"</span><span class="p">))</span>
|
||||
</span><span id="L-373"><a href="#L-373"><span class="linenos">373</span></a>
|
||||
</span><span id="L-355"><a href="#L-355"><span class="linenos">355</span></a>
|
||||
</span><span id="L-356"><a href="#L-356"><span class="linenos">356</span></a><span class="k">def</span> <span class="nf">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-357"><a href="#L-357"><span class="linenos">357</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-358"><a href="#L-358"><span class="linenos">358</span></a><span class="sd"> Merge ORDER clause of inner query into outer query.</span>
|
||||
</span><span id="L-359"><a href="#L-359"><span class="linenos">359</span></a>
|
||||
</span><span id="L-360"><a href="#L-360"><span class="linenos">360</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-361"><a href="#L-361"><span class="linenos">361</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-362"><a href="#L-362"><span class="linenos">362</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-363"><a href="#L-363"><span class="linenos">363</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-364"><a href="#L-364"><span class="linenos">364</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-365"><a href="#L-365"><span class="linenos">365</span></a> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-366"><a href="#L-366"><span class="linenos">366</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="p">[</span><span class="s2">"group"</span><span class="p">,</span> <span class="s2">"distinct"</span><span class="p">,</span> <span class="s2">"having"</span><span class="p">,</span> <span class="s2">"order"</span><span class="p">]</span>
|
||||
</span><span id="L-367"><a href="#L-367"><span class="linenos">367</span></a> <span class="p">)</span>
|
||||
</span><span id="L-368"><a href="#L-368"><span class="linenos">368</span></a> <span class="ow">or</span> <span class="nb">len</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span> <span class="o">!=</span> <span class="mi">1</span>
|
||||
</span><span id="L-369"><a href="#L-369"><span class="linenos">369</span></a> <span class="ow">or</span> <span class="nb">any</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">)</span> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span>
|
||||
</span><span id="L-370"><a href="#L-370"><span class="linenos">370</span></a> <span class="p">):</span>
|
||||
</span><span id="L-371"><a href="#L-371"><span class="linenos">371</span></a> <span class="k">return</span>
|
||||
</span><span id="L-372"><a href="#L-372"><span class="linenos">372</span></a>
|
||||
</span><span id="L-373"><a href="#L-373"><span class="linenos">373</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"order"</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"order"</span><span class="p">))</span>
|
||||
</span><span id="L-374"><a href="#L-374"><span class="linenos">374</span></a>
|
||||
</span><span id="L-375"><a href="#L-375"><span class="linenos">375</span></a><span class="k">def</span> <span class="nf">_merge_hints</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-376"><a href="#L-376"><span class="linenos">376</span></a> <span class="n">inner_scope_hint</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">)</span>
|
||||
</span><span id="L-377"><a href="#L-377"><span class="linenos">377</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">inner_scope_hint</span><span class="p">:</span>
|
||||
</span><span id="L-378"><a href="#L-378"><span class="linenos">378</span></a> <span class="k">return</span>
|
||||
</span><span id="L-379"><a href="#L-379"><span class="linenos">379</span></a> <span class="n">outer_scope_hint</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">)</span>
|
||||
</span><span id="L-380"><a href="#L-380"><span class="linenos">380</span></a> <span class="k">if</span> <span class="n">outer_scope_hint</span><span class="p">:</span>
|
||||
</span><span id="L-381"><a href="#L-381"><span class="linenos">381</span></a> <span class="k">for</span> <span class="n">hint_expression</span> <span class="ow">in</span> <span class="n">inner_scope_hint</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-382"><a href="#L-382"><span class="linenos">382</span></a> <span class="n">outer_scope_hint</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"expressions"</span><span class="p">,</span> <span class="n">hint_expression</span><span class="p">)</span>
|
||||
</span><span id="L-383"><a href="#L-383"><span class="linenos">383</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-384"><a href="#L-384"><span class="linenos">384</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">,</span> <span class="n">inner_scope_hint</span><span class="p">)</span>
|
||||
</span><span id="L-385"><a href="#L-385"><span class="linenos">385</span></a>
|
||||
</span><span id="L-375"><a href="#L-375"><span class="linenos">375</span></a>
|
||||
</span><span id="L-376"><a href="#L-376"><span class="linenos">376</span></a><span class="k">def</span> <span class="nf">_merge_hints</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-377"><a href="#L-377"><span class="linenos">377</span></a> <span class="n">inner_scope_hint</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">)</span>
|
||||
</span><span id="L-378"><a href="#L-378"><span class="linenos">378</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">inner_scope_hint</span><span class="p">:</span>
|
||||
</span><span id="L-379"><a href="#L-379"><span class="linenos">379</span></a> <span class="k">return</span>
|
||||
</span><span id="L-380"><a href="#L-380"><span class="linenos">380</span></a> <span class="n">outer_scope_hint</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">)</span>
|
||||
</span><span id="L-381"><a href="#L-381"><span class="linenos">381</span></a> <span class="k">if</span> <span class="n">outer_scope_hint</span><span class="p">:</span>
|
||||
</span><span id="L-382"><a href="#L-382"><span class="linenos">382</span></a> <span class="k">for</span> <span class="n">hint_expression</span> <span class="ow">in</span> <span class="n">inner_scope_hint</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-383"><a href="#L-383"><span class="linenos">383</span></a> <span class="n">outer_scope_hint</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"expressions"</span><span class="p">,</span> <span class="n">hint_expression</span><span class="p">)</span>
|
||||
</span><span id="L-384"><a href="#L-384"><span class="linenos">384</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-385"><a href="#L-385"><span class="linenos">385</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">,</span> <span class="n">inner_scope_hint</span><span class="p">)</span>
|
||||
</span><span id="L-386"><a href="#L-386"><span class="linenos">386</span></a>
|
||||
</span><span id="L-387"><a href="#L-387"><span class="linenos">387</span></a><span class="k">def</span> <span class="nf">_pop_cte</span><span class="p">(</span><span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-388"><a href="#L-388"><span class="linenos">388</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-389"><a href="#L-389"><span class="linenos">389</span></a><span class="sd"> Remove CTE from the AST.</span>
|
||||
</span><span id="L-390"><a href="#L-390"><span class="linenos">390</span></a>
|
||||
</span><span id="L-391"><a href="#L-391"><span class="linenos">391</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-392"><a href="#L-392"><span class="linenos">392</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-393"><a href="#L-393"><span class="linenos">393</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-394"><a href="#L-394"><span class="linenos">394</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-395"><a href="#L-395"><span class="linenos">395</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">cte</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-396"><a href="#L-396"><span class="linenos">396</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">with_</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-397"><a href="#L-397"><span class="linenos">397</span></a> <span class="n">with_</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-398"><a href="#L-398"><span class="linenos">398</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-399"><a href="#L-399"><span class="linenos">399</span></a> <span class="n">cte</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-387"><a href="#L-387"><span class="linenos">387</span></a>
|
||||
</span><span id="L-388"><a href="#L-388"><span class="linenos">388</span></a><span class="k">def</span> <span class="nf">_pop_cte</span><span class="p">(</span><span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-389"><a href="#L-389"><span class="linenos">389</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-390"><a href="#L-390"><span class="linenos">390</span></a><span class="sd"> Remove CTE from the AST.</span>
|
||||
</span><span id="L-391"><a href="#L-391"><span class="linenos">391</span></a>
|
||||
</span><span id="L-392"><a href="#L-392"><span class="linenos">392</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-393"><a href="#L-393"><span class="linenos">393</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-394"><a href="#L-394"><span class="linenos">394</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-395"><a href="#L-395"><span class="linenos">395</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-396"><a href="#L-396"><span class="linenos">396</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">cte</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-397"><a href="#L-397"><span class="linenos">397</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">with_</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-398"><a href="#L-398"><span class="linenos">398</span></a> <span class="n">with_</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-399"><a href="#L-399"><span class="linenos">399</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-400"><a href="#L-400"><span class="linenos">400</span></a> <span class="n">cte</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -565,7 +566,7 @@ queries if it would result in multiple table selects in a single query:</p>
|
|||
<div class="attr variable">
|
||||
<span class="name">UNMERGABLE_ARGS</span> =
|
||||
<input id="UNMERGABLE_ARGS-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{'sample', 'group', 'having', 'windows', 'offset', 'into', 'qualify', 'pivots', 'match', 'laterals', 'settings', 'cluster', 'format', 'limit', 'distinct', 'locks', 'distribute', 'with', 'kind', 'sort'}</span>
|
||||
<label class="view-value-button pdoc-button" for="UNMERGABLE_ARGS-view-value"></label><span class="default_value">{'into', 'windows', 'cluster', 'match', 'distinct', 'group', 'with', 'distribute', 'locks', 'laterals', 'sample', 'having', 'limit', 'sort', 'pivots', 'kind', 'qualify', 'settings', 'offset', 'format'}</span>
|
||||
|
||||
|
||||
</div>
|
||||
|
|
@ -662,7 +663,8 @@ queries if it would result in multiple table selects in a single query:</p>
|
|||
</span><span id="merge_derived_tables-108"><a href="#merge_derived_tables-108"><span class="linenos">108</span></a> <span class="n">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
|
||||
</span><span id="merge_derived_tables-109"><a href="#merge_derived_tables-109"><span class="linenos">109</span></a> <span class="n">_merge_hints</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">)</span>
|
||||
</span><span id="merge_derived_tables-110"><a href="#merge_derived_tables-110"><span class="linenos">110</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">clear_cache</span><span class="p">()</span>
|
||||
</span><span id="merge_derived_tables-111"><a href="#merge_derived_tables-111"><span class="linenos">111</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="merge_derived_tables-111"><a href="#merge_derived_tables-111"><span class="linenos">111</span></a>
|
||||
</span><span id="merge_derived_tables-112"><a href="#merge_derived_tables-112"><span class="linenos">112</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -129,7 +129,7 @@
|
|||
</span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a> <span class="n">dag</span> <span class="o">=</span> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">)</span> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="o">.</span><span class="n">items</span><span class="p">()}</span>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a> <span class="s2">"joins"</span><span class="p">,</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a> <span class="p">[</span><span class="n">joins</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">tsort</span><span class="p">(</span><span class="n">dag</span><span class="p">)</span> <span class="k">if</span> <span class="n">name</span> <span class="o">!=</span> <span class="n">from_</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">],</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a> <span class="p">[</span><span class="n">joins</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">tsort</span><span class="p">(</span><span class="n">dag</span><span class="p">)</span> <span class="k">if</span> <span class="n">name</span> <span class="o">!=</span> <span class="n">from_</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">and</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">joins</span><span class="p">],</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos">63</span></a> <span class="p">)</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos">64</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos">65</span></a>
|
||||
|
|
@ -256,7 +256,7 @@
|
|||
</span><span id="reorder_joins-60"><a href="#reorder_joins-60"><span class="linenos">60</span></a> <span class="n">dag</span> <span class="o">=</span> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">)</span> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="o">.</span><span class="n">items</span><span class="p">()}</span>
|
||||
</span><span id="reorder_joins-61"><a href="#reorder_joins-61"><span class="linenos">61</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="reorder_joins-62"><a href="#reorder_joins-62"><span class="linenos">62</span></a> <span class="s2">"joins"</span><span class="p">,</span>
|
||||
</span><span id="reorder_joins-63"><a href="#reorder_joins-63"><span class="linenos">63</span></a> <span class="p">[</span><span class="n">joins</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">tsort</span><span class="p">(</span><span class="n">dag</span><span class="p">)</span> <span class="k">if</span> <span class="n">name</span> <span class="o">!=</span> <span class="n">from_</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">],</span>
|
||||
</span><span id="reorder_joins-63"><a href="#reorder_joins-63"><span class="linenos">63</span></a> <span class="p">[</span><span class="n">joins</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">tsort</span><span class="p">(</span><span class="n">dag</span><span class="p">)</span> <span class="k">if</span> <span class="n">name</span> <span class="o">!=</span> <span class="n">from_</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">and</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">joins</span><span class="p">],</span>
|
||||
</span><span id="reorder_joins-64"><a href="#reorder_joins-64"><span class="linenos">64</span></a> <span class="p">)</span>
|
||||
</span><span id="reorder_joins-65"><a href="#reorder_joins-65"><span class="linenos">65</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -115,161 +115,164 @@
|
|||
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="c1"># so we limit the selected sources to only itself</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]:</span>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="n">pushdown</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">),</span> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">name</span><span class="p">]},</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">:</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="n">pushdown</span><span class="p">(</span>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">),</span> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">name</span><span class="p">]},</span> <span class="n">scope_ref_count</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="p">)</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a><span class="k">def</span> <span class="nf">pushdown</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">condition</span><span class="p">:</span>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">return</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">condition</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">simplify</span><span class="p">(</span><span class="n">condition</span><span class="p">))</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">cnf_like</span> <span class="o">=</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">)</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a><span class="k">def</span> <span class="nf">pushdown</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">condition</span><span class="p">:</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="k">return</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="n">predicates</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">condition</span><span class="o">.</span><span class="n">flatten</span><span class="p">()</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">if</span> <span class="n">cnf_like</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="k">else</span> <span class="p">[</span><span class="n">condition</span><span class="p">]</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="p">)</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="k">if</span> <span class="n">cnf_like</span><span class="p">:</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="n">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="n">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a><span class="k">def</span> <span class="nf">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a><span class="sd"> If the predicates are in CNF like form, we can simply replace each block in the parent.</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">break</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a><span class="k">def</span> <span class="nf">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a><span class="sd"> If the predicates are in DNF form, we can only push down conditions that are in all blocks.</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a><span class="sd"> Additionally, we can't remove predicates from their original form.</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="c1"># find all the tables that can be pushdown too</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="c1"># these are tables that are referenced in all blocks of a DNF</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="c1"># (a.x AND b.x) OR (a.y AND c.y)</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="c1"># only table a can be push down</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="n">pushdown_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">condition</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">simplify</span><span class="p">(</span><span class="n">condition</span><span class="p">))</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">cnf_like</span> <span class="o">=</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">)</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="n">predicates</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="n">condition</span><span class="o">.</span><span class="n">flatten</span><span class="p">()</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">if</span> <span class="n">cnf_like</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="k">else</span> <span class="p">[</span><span class="n">condition</span><span class="p">]</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="p">)</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="k">if</span> <span class="n">cnf_like</span><span class="p">:</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="n">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="n">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a><span class="k">def</span> <span class="nf">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a><span class="sd"> If the predicates are in CNF like form, we can simply replace each block in the parent.</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="k">break</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a><span class="k">def</span> <span class="nf">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a><span class="sd"> If the predicates are in DNF form, we can only push down conditions that are in all blocks.</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a><span class="sd"> Additionally, we can't remove predicates from their original form.</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="c1"># find all the tables that can be pushdown too</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="c1"># these are tables that are referenced in all blocks of a DNF</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="c1"># (a.x AND b.x) OR (a.y AND c.y)</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="c1"># only table a can be push down</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">pushdown_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">a_tables</span> <span class="o">&=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">pushdown_tables</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">a_tables</span><span class="p">)</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">conditions</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="c1"># for every pushdown table, find all related conditions in all predicates</span>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="c1"># combine them with ORS</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="c1"># (a.x AND and a.y AND b.x) OR (a.z AND c.y) -> (a.x AND a.y) OR (a.z)</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">pushdown_tables</span><span class="p">):</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">nodes</span><span class="p">:</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="n">a_tables</span> <span class="o">&=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="n">pushdown_tables</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">a_tables</span><span class="p">)</span>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="n">conditions</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="c1"># for every pushdown table, find all related conditions in all predicates</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="c1"># combine them with ORS</span>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="c1"># (a.x AND and a.y AND b.x) OR (a.z AND c.y) -> (a.x AND a.y) OR (a.z)</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">pushdown_tables</span><span class="p">):</span>
|
||||
</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">predicate</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">):</span>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">table</span><span class="p">:</span>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">column</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Condition</span><span class="p">)</span>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span><span class="p">(</span><span class="n">predicate_condition</span><span class="p">,</span> <span class="n">condition</span><span class="p">)</span>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="k">if</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="k">else</span> <span class="n">condition</span>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="p">)</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="k">if</span> <span class="n">predicate_condition</span><span class="p">:</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span><span class="p">(</span><span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">],</span> <span class="n">predicate_condition</span><span class="p">)</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">conditions</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="k">else</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="p">)</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">conditions</span><span class="p">:</span>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="n">predicate</span> <span class="o">=</span> <span class="n">conditions</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a>
|
||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a><span class="k">def</span> <span class="nf">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">where_condition</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">)</span>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">tables</span><span class="p">):</span>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">source</span> <span class="o">=</span> <span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="ow">or</span> <span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">nodes</span><span class="p">:</span>
|
||||
</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a>
|
||||
</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a>
|
||||
</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">predicate</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">):</span>
|
||||
</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">table</span><span class="p">:</span>
|
||||
</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">column</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Condition</span><span class="p">)</span>
|
||||
</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span><span class="p">(</span><span class="n">predicate_condition</span><span class="p">,</span> <span class="n">condition</span><span class="p">)</span>
|
||||
</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="k">if</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="k">else</span> <span class="n">condition</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="p">)</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="k">if</span> <span class="n">predicate_condition</span><span class="p">:</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span><span class="p">(</span><span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">],</span> <span class="n">predicate_condition</span><span class="p">)</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">conditions</span>
|
||||
</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="k">else</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="p">)</span>
|
||||
</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a>
|
||||
</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">conditions</span><span class="p">:</span>
|
||||
</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a>
|
||||
</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">predicate</span> <span class="o">=</span> <span class="n">conditions</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
|
||||
</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a>
|
||||
</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a><span class="k">def</span> <span class="nf">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="n">where_condition</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">)</span>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="c1"># if the predicate is in a where statement we can try to push it down</span>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="c1"># we want to find the root join or from statement</span>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">where_condition</span><span class="p">:</span>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="c1"># a node can reference a CTE which should be pushed down</span>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"with"</span><span class="p">)</span>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="k">if</span> <span class="n">with_</span> <span class="ow">and</span> <span class="n">with_</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="k">if</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="o">!=</span> <span class="s2">"RIGHT"</span><span class="p">:</span>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="c1"># We can't push down window expressions</span>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="n">has_window_expression</span> <span class="o">=</span> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="p">)</span>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="c1"># we can't push down predicates to select statements if they are referenced in</span>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="c1"># multiple places.</span>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"group"</span><span class="p">)</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="ow">and</span> <span class="n">scope_ref_count</span><span class="p">[</span><span class="nb">id</span><span class="p">(</span><span class="n">source</span><span class="p">)]</span> <span class="o"><</span> <span class="mi">2</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">has_window_expression</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="p">):</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="k">return</span> <span class="n">nodes</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a><span class="k">def</span> <span class="nf">replace_aliases</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">predicate</span><span class="p">):</span>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="n">aliases</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">tables</span><span class="p">):</span>
|
||||
</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">source</span> <span class="o">=</span> <span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="ow">or</span> <span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a>
|
||||
</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="c1"># if the predicate is in a where statement we can try to push it down</span>
|
||||
</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="c1"># we want to find the root join or from statement</span>
|
||||
</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">where_condition</span><span class="p">:</span>
|
||||
</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
|
||||
</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a>
|
||||
</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="c1"># a node can reference a CTE which should be pushed down</span>
|
||||
</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"with"</span><span class="p">)</span>
|
||||
</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="k">if</span> <span class="n">with_</span> <span class="ow">and</span> <span class="n">with_</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
|
||||
</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a>
|
||||
</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="k">if</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="o">!=</span> <span class="s2">"RIGHT"</span><span class="p">:</span>
|
||||
</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="c1"># We can't push down window expressions</span>
|
||||
</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="n">has_window_expression</span> <span class="o">=</span> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
|
||||
</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="p">)</span>
|
||||
</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="c1"># we can't push down predicates to select statements if they are referenced in</span>
|
||||
</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="c1"># multiple places.</span>
|
||||
</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"group"</span><span class="p">)</span>
|
||||
</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="ow">and</span> <span class="n">scope_ref_count</span><span class="p">[</span><span class="nb">id</span><span class="p">(</span><span class="n">source</span><span class="p">)]</span> <span class="o"><</span> <span class="mi">2</span>
|
||||
</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">has_window_expression</span>
|
||||
</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="p">):</span>
|
||||
</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="k">return</span> <span class="n">nodes</span>
|
||||
</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a>
|
||||
</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="k">def</span> <span class="nf">_replace_alias</span><span class="p">(</span><span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="ow">and</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">aliases</span><span class="p">:</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="k">return</span> <span class="n">aliases</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="k">return</span> <span class="n">column</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="k">return</span> <span class="n">predicate</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_replace_alias</span><span class="p">)</span>
|
||||
</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a><span class="k">def</span> <span class="nf">replace_aliases</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">predicate</span><span class="p">):</span>
|
||||
</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="n">aliases</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a>
|
||||
</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="k">def</span> <span class="nf">_replace_alias</span><span class="p">(</span><span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="ow">and</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">aliases</span><span class="p">:</span>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="k">return</span> <span class="n">aliases</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="k">return</span> <span class="n">column</span>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a>
|
||||
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="k">return</span> <span class="n">predicate</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_replace_alias</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -323,9 +326,12 @@
|
|||
</span><span id="pushdown_predicates-43"><a href="#pushdown_predicates-43"><span class="linenos">43</span></a> <span class="c1"># so we limit the selected sources to only itself</span>
|
||||
</span><span id="pushdown_predicates-44"><a href="#pushdown_predicates-44"><span class="linenos">44</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]:</span>
|
||||
</span><span id="pushdown_predicates-45"><a href="#pushdown_predicates-45"><span class="linenos">45</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="pushdown_predicates-46"><a href="#pushdown_predicates-46"><span class="linenos">46</span></a> <span class="n">pushdown</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">),</span> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">name</span><span class="p">]},</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="pushdown_predicates-47"><a href="#pushdown_predicates-47"><span class="linenos">47</span></a>
|
||||
</span><span id="pushdown_predicates-48"><a href="#pushdown_predicates-48"><span class="linenos">48</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="pushdown_predicates-46"><a href="#pushdown_predicates-46"><span class="linenos">46</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">:</span>
|
||||
</span><span id="pushdown_predicates-47"><a href="#pushdown_predicates-47"><span class="linenos">47</span></a> <span class="n">pushdown</span><span class="p">(</span>
|
||||
</span><span id="pushdown_predicates-48"><a href="#pushdown_predicates-48"><span class="linenos">48</span></a> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">),</span> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">name</span><span class="p">]},</span> <span class="n">scope_ref_count</span>
|
||||
</span><span id="pushdown_predicates-49"><a href="#pushdown_predicates-49"><span class="linenos">49</span></a> <span class="p">)</span>
|
||||
</span><span id="pushdown_predicates-50"><a href="#pushdown_predicates-50"><span class="linenos">50</span></a>
|
||||
</span><span id="pushdown_predicates-51"><a href="#pushdown_predicates-51"><span class="linenos">51</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -370,23 +376,23 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#pushdown"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown-51"><a href="#pushdown-51"><span class="linenos">51</span></a><span class="k">def</span> <span class="nf">pushdown</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="pushdown-52"><a href="#pushdown-52"><span class="linenos">52</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">condition</span><span class="p">:</span>
|
||||
</span><span id="pushdown-53"><a href="#pushdown-53"><span class="linenos">53</span></a> <span class="k">return</span>
|
||||
</span><span id="pushdown-54"><a href="#pushdown-54"><span class="linenos">54</span></a>
|
||||
</span><span id="pushdown-55"><a href="#pushdown-55"><span class="linenos">55</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">condition</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">simplify</span><span class="p">(</span><span class="n">condition</span><span class="p">))</span>
|
||||
</span><span id="pushdown-56"><a href="#pushdown-56"><span class="linenos">56</span></a> <span class="n">cnf_like</span> <span class="o">=</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">)</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown-54"><a href="#pushdown-54"><span class="linenos">54</span></a><span class="k">def</span> <span class="nf">pushdown</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="pushdown-55"><a href="#pushdown-55"><span class="linenos">55</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">condition</span><span class="p">:</span>
|
||||
</span><span id="pushdown-56"><a href="#pushdown-56"><span class="linenos">56</span></a> <span class="k">return</span>
|
||||
</span><span id="pushdown-57"><a href="#pushdown-57"><span class="linenos">57</span></a>
|
||||
</span><span id="pushdown-58"><a href="#pushdown-58"><span class="linenos">58</span></a> <span class="n">predicates</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span>
|
||||
</span><span id="pushdown-59"><a href="#pushdown-59"><span class="linenos">59</span></a> <span class="n">condition</span><span class="o">.</span><span class="n">flatten</span><span class="p">()</span>
|
||||
</span><span id="pushdown-60"><a href="#pushdown-60"><span class="linenos">60</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">if</span> <span class="n">cnf_like</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span>
|
||||
</span><span id="pushdown-61"><a href="#pushdown-61"><span class="linenos">61</span></a> <span class="k">else</span> <span class="p">[</span><span class="n">condition</span><span class="p">]</span>
|
||||
</span><span id="pushdown-62"><a href="#pushdown-62"><span class="linenos">62</span></a> <span class="p">)</span>
|
||||
</span><span id="pushdown-63"><a href="#pushdown-63"><span class="linenos">63</span></a>
|
||||
</span><span id="pushdown-64"><a href="#pushdown-64"><span class="linenos">64</span></a> <span class="k">if</span> <span class="n">cnf_like</span><span class="p">:</span>
|
||||
</span><span id="pushdown-65"><a href="#pushdown-65"><span class="linenos">65</span></a> <span class="n">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="pushdown-66"><a href="#pushdown-66"><span class="linenos">66</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="pushdown-67"><a href="#pushdown-67"><span class="linenos">67</span></a> <span class="n">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="pushdown-58"><a href="#pushdown-58"><span class="linenos">58</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">condition</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">simplify</span><span class="p">(</span><span class="n">condition</span><span class="p">))</span>
|
||||
</span><span id="pushdown-59"><a href="#pushdown-59"><span class="linenos">59</span></a> <span class="n">cnf_like</span> <span class="o">=</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">)</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">normalized</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">dnf</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="pushdown-60"><a href="#pushdown-60"><span class="linenos">60</span></a>
|
||||
</span><span id="pushdown-61"><a href="#pushdown-61"><span class="linenos">61</span></a> <span class="n">predicates</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span>
|
||||
</span><span id="pushdown-62"><a href="#pushdown-62"><span class="linenos">62</span></a> <span class="n">condition</span><span class="o">.</span><span class="n">flatten</span><span class="p">()</span>
|
||||
</span><span id="pushdown-63"><a href="#pushdown-63"><span class="linenos">63</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">condition</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">And</span> <span class="k">if</span> <span class="n">cnf_like</span> <span class="k">else</span> <span class="n">exp</span><span class="o">.</span><span class="n">Or</span><span class="p">)</span>
|
||||
</span><span id="pushdown-64"><a href="#pushdown-64"><span class="linenos">64</span></a> <span class="k">else</span> <span class="p">[</span><span class="n">condition</span><span class="p">]</span>
|
||||
</span><span id="pushdown-65"><a href="#pushdown-65"><span class="linenos">65</span></a> <span class="p">)</span>
|
||||
</span><span id="pushdown-66"><a href="#pushdown-66"><span class="linenos">66</span></a>
|
||||
</span><span id="pushdown-67"><a href="#pushdown-67"><span class="linenos">67</span></a> <span class="k">if</span> <span class="n">cnf_like</span><span class="p">:</span>
|
||||
</span><span id="pushdown-68"><a href="#pushdown-68"><span class="linenos">68</span></a> <span class="n">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="pushdown-69"><a href="#pushdown-69"><span class="linenos">69</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="pushdown-70"><a href="#pushdown-70"><span class="linenos">70</span></a> <span class="n">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -404,19 +410,19 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#pushdown_cnf"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown_cnf-70"><a href="#pushdown_cnf-70"><span class="linenos">70</span></a><span class="k">def</span> <span class="nf">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="pushdown_cnf-71"><a href="#pushdown_cnf-71"><span class="linenos">71</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="pushdown_cnf-72"><a href="#pushdown_cnf-72"><span class="linenos">72</span></a><span class="sd"> If the predicates are in CNF like form, we can simply replace each block in the parent.</span>
|
||||
</span><span id="pushdown_cnf-73"><a href="#pushdown_cnf-73"><span class="linenos">73</span></a><span class="sd"> """</span>
|
||||
</span><span id="pushdown_cnf-74"><a href="#pushdown_cnf-74"><span class="linenos">74</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_cnf-75"><a href="#pushdown_cnf-75"><span class="linenos">75</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
|
||||
</span><span id="pushdown_cnf-76"><a href="#pushdown_cnf-76"><span class="linenos">76</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="pushdown_cnf-77"><a href="#pushdown_cnf-77"><span class="linenos">77</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="pushdown_cnf-78"><a href="#pushdown_cnf-78"><span class="linenos">78</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="pushdown_cnf-79"><a href="#pushdown_cnf-79"><span class="linenos">79</span></a> <span class="k">break</span>
|
||||
</span><span id="pushdown_cnf-80"><a href="#pushdown_cnf-80"><span class="linenos">80</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="pushdown_cnf-81"><a href="#pushdown_cnf-81"><span class="linenos">81</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="pushdown_cnf-82"><a href="#pushdown_cnf-82"><span class="linenos">82</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown_cnf-73"><a href="#pushdown_cnf-73"><span class="linenos">73</span></a><span class="k">def</span> <span class="nf">pushdown_cnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="pushdown_cnf-74"><a href="#pushdown_cnf-74"><span class="linenos">74</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="pushdown_cnf-75"><a href="#pushdown_cnf-75"><span class="linenos">75</span></a><span class="sd"> If the predicates are in CNF like form, we can simply replace each block in the parent.</span>
|
||||
</span><span id="pushdown_cnf-76"><a href="#pushdown_cnf-76"><span class="linenos">76</span></a><span class="sd"> """</span>
|
||||
</span><span id="pushdown_cnf-77"><a href="#pushdown_cnf-77"><span class="linenos">77</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_cnf-78"><a href="#pushdown_cnf-78"><span class="linenos">78</span></a> <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span><span class="o">.</span><span class="n">values</span><span class="p">():</span>
|
||||
</span><span id="pushdown_cnf-79"><a href="#pushdown_cnf-79"><span class="linenos">79</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="pushdown_cnf-80"><a href="#pushdown_cnf-80"><span class="linenos">80</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="pushdown_cnf-81"><a href="#pushdown_cnf-81"><span class="linenos">81</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="pushdown_cnf-82"><a href="#pushdown_cnf-82"><span class="linenos">82</span></a> <span class="k">break</span>
|
||||
</span><span id="pushdown_cnf-83"><a href="#pushdown_cnf-83"><span class="linenos">83</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="pushdown_cnf-84"><a href="#pushdown_cnf-84"><span class="linenos">84</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="pushdown_cnf-85"><a href="#pushdown_cnf-85"><span class="linenos">85</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -436,65 +442,65 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#pushdown_dnf"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown_dnf-85"><a href="#pushdown_dnf-85"><span class="linenos"> 85</span></a><span class="k">def</span> <span class="nf">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-86"><a href="#pushdown_dnf-86"><span class="linenos"> 86</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="pushdown_dnf-87"><a href="#pushdown_dnf-87"><span class="linenos"> 87</span></a><span class="sd"> If the predicates are in DNF form, we can only push down conditions that are in all blocks.</span>
|
||||
</span><span id="pushdown_dnf-88"><a href="#pushdown_dnf-88"><span class="linenos"> 88</span></a><span class="sd"> Additionally, we can't remove predicates from their original form.</span>
|
||||
</span><span id="pushdown_dnf-89"><a href="#pushdown_dnf-89"><span class="linenos"> 89</span></a><span class="sd"> """</span>
|
||||
</span><span id="pushdown_dnf-90"><a href="#pushdown_dnf-90"><span class="linenos"> 90</span></a> <span class="c1"># find all the tables that can be pushdown too</span>
|
||||
</span><span id="pushdown_dnf-91"><a href="#pushdown_dnf-91"><span class="linenos"> 91</span></a> <span class="c1"># these are tables that are referenced in all blocks of a DNF</span>
|
||||
</span><span id="pushdown_dnf-92"><a href="#pushdown_dnf-92"><span class="linenos"> 92</span></a> <span class="c1"># (a.x AND b.x) OR (a.y AND c.y)</span>
|
||||
</span><span id="pushdown_dnf-93"><a href="#pushdown_dnf-93"><span class="linenos"> 93</span></a> <span class="c1"># only table a can be push down</span>
|
||||
</span><span id="pushdown_dnf-94"><a href="#pushdown_dnf-94"><span class="linenos"> 94</span></a> <span class="n">pushdown_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="pushdown_dnf-95"><a href="#pushdown_dnf-95"><span class="linenos"> 95</span></a>
|
||||
</span><span id="pushdown_dnf-96"><a href="#pushdown_dnf-96"><span class="linenos"> 96</span></a> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-97"><a href="#pushdown_dnf-97"><span class="linenos"> 97</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="pushdown_dnf-88"><a href="#pushdown_dnf-88"><span class="linenos"> 88</span></a><span class="k">def</span> <span class="nf">pushdown_dnf</span><span class="p">(</span><span class="n">predicates</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-89"><a href="#pushdown_dnf-89"><span class="linenos"> 89</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="pushdown_dnf-90"><a href="#pushdown_dnf-90"><span class="linenos"> 90</span></a><span class="sd"> If the predicates are in DNF form, we can only push down conditions that are in all blocks.</span>
|
||||
</span><span id="pushdown_dnf-91"><a href="#pushdown_dnf-91"><span class="linenos"> 91</span></a><span class="sd"> Additionally, we can't remove predicates from their original form.</span>
|
||||
</span><span id="pushdown_dnf-92"><a href="#pushdown_dnf-92"><span class="linenos"> 92</span></a><span class="sd"> """</span>
|
||||
</span><span id="pushdown_dnf-93"><a href="#pushdown_dnf-93"><span class="linenos"> 93</span></a> <span class="c1"># find all the tables that can be pushdown too</span>
|
||||
</span><span id="pushdown_dnf-94"><a href="#pushdown_dnf-94"><span class="linenos"> 94</span></a> <span class="c1"># these are tables that are referenced in all blocks of a DNF</span>
|
||||
</span><span id="pushdown_dnf-95"><a href="#pushdown_dnf-95"><span class="linenos"> 95</span></a> <span class="c1"># (a.x AND b.x) OR (a.y AND c.y)</span>
|
||||
</span><span id="pushdown_dnf-96"><a href="#pushdown_dnf-96"><span class="linenos"> 96</span></a> <span class="c1"># only table a can be push down</span>
|
||||
</span><span id="pushdown_dnf-97"><a href="#pushdown_dnf-97"><span class="linenos"> 97</span></a> <span class="n">pushdown_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="pushdown_dnf-98"><a href="#pushdown_dnf-98"><span class="linenos"> 98</span></a>
|
||||
</span><span id="pushdown_dnf-99"><a href="#pushdown_dnf-99"><span class="linenos"> 99</span></a> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-100"><a href="#pushdown_dnf-100"><span class="linenos">100</span></a> <span class="n">a_tables</span> <span class="o">&=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-99"><a href="#pushdown_dnf-99"><span class="linenos"> 99</span></a> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-100"><a href="#pushdown_dnf-100"><span class="linenos">100</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-101"><a href="#pushdown_dnf-101"><span class="linenos">101</span></a>
|
||||
</span><span id="pushdown_dnf-102"><a href="#pushdown_dnf-102"><span class="linenos">102</span></a> <span class="n">pushdown_tables</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">a_tables</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-103"><a href="#pushdown_dnf-103"><span class="linenos">103</span></a>
|
||||
</span><span id="pushdown_dnf-104"><a href="#pushdown_dnf-104"><span class="linenos">104</span></a> <span class="n">conditions</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="pushdown_dnf-105"><a href="#pushdown_dnf-105"><span class="linenos">105</span></a>
|
||||
</span><span id="pushdown_dnf-106"><a href="#pushdown_dnf-106"><span class="linenos">106</span></a> <span class="c1"># for every pushdown table, find all related conditions in all predicates</span>
|
||||
</span><span id="pushdown_dnf-107"><a href="#pushdown_dnf-107"><span class="linenos">107</span></a> <span class="c1"># combine them with ORS</span>
|
||||
</span><span id="pushdown_dnf-108"><a href="#pushdown_dnf-108"><span class="linenos">108</span></a> <span class="c1"># (a.x AND and a.y AND b.x) OR (a.z AND c.y) -> (a.x AND a.y) OR (a.z)</span>
|
||||
</span><span id="pushdown_dnf-109"><a href="#pushdown_dnf-109"><span class="linenos">109</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">pushdown_tables</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-110"><a href="#pushdown_dnf-110"><span class="linenos">110</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-111"><a href="#pushdown_dnf-111"><span class="linenos">111</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-112"><a href="#pushdown_dnf-112"><span class="linenos">112</span></a>
|
||||
</span><span id="pushdown_dnf-113"><a href="#pushdown_dnf-113"><span class="linenos">113</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">nodes</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-114"><a href="#pushdown_dnf-114"><span class="linenos">114</span></a> <span class="k">continue</span>
|
||||
</span><span id="pushdown_dnf-102"><a href="#pushdown_dnf-102"><span class="linenos">102</span></a> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-103"><a href="#pushdown_dnf-103"><span class="linenos">103</span></a> <span class="n">a_tables</span> <span class="o">&=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-104"><a href="#pushdown_dnf-104"><span class="linenos">104</span></a>
|
||||
</span><span id="pushdown_dnf-105"><a href="#pushdown_dnf-105"><span class="linenos">105</span></a> <span class="n">pushdown_tables</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">a_tables</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-106"><a href="#pushdown_dnf-106"><span class="linenos">106</span></a>
|
||||
</span><span id="pushdown_dnf-107"><a href="#pushdown_dnf-107"><span class="linenos">107</span></a> <span class="n">conditions</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="pushdown_dnf-108"><a href="#pushdown_dnf-108"><span class="linenos">108</span></a>
|
||||
</span><span id="pushdown_dnf-109"><a href="#pushdown_dnf-109"><span class="linenos">109</span></a> <span class="c1"># for every pushdown table, find all related conditions in all predicates</span>
|
||||
</span><span id="pushdown_dnf-110"><a href="#pushdown_dnf-110"><span class="linenos">110</span></a> <span class="c1"># combine them with ORS</span>
|
||||
</span><span id="pushdown_dnf-111"><a href="#pushdown_dnf-111"><span class="linenos">111</span></a> <span class="c1"># (a.x AND and a.y AND b.x) OR (a.z AND c.y) -> (a.x AND a.y) OR (a.z)</span>
|
||||
</span><span id="pushdown_dnf-112"><a href="#pushdown_dnf-112"><span class="linenos">112</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">pushdown_tables</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-113"><a href="#pushdown_dnf-113"><span class="linenos">113</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-114"><a href="#pushdown_dnf-114"><span class="linenos">114</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="n">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-115"><a href="#pushdown_dnf-115"><span class="linenos">115</span></a>
|
||||
</span><span id="pushdown_dnf-116"><a href="#pushdown_dnf-116"><span class="linenos">116</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="pushdown_dnf-117"><a href="#pushdown_dnf-117"><span class="linenos">117</span></a>
|
||||
</span><span id="pushdown_dnf-118"><a href="#pushdown_dnf-118"><span class="linenos">118</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">predicate</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-119"><a href="#pushdown_dnf-119"><span class="linenos">119</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">table</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-120"><a href="#pushdown_dnf-120"><span class="linenos">120</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">column</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Condition</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-121"><a href="#pushdown_dnf-121"><span class="linenos">121</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="pushdown_dnf-122"><a href="#pushdown_dnf-122"><span class="linenos">122</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span><span class="p">(</span><span class="n">predicate_condition</span><span class="p">,</span> <span class="n">condition</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-123"><a href="#pushdown_dnf-123"><span class="linenos">123</span></a> <span class="k">if</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="pushdown_dnf-124"><a href="#pushdown_dnf-124"><span class="linenos">124</span></a> <span class="k">else</span> <span class="n">condition</span>
|
||||
</span><span id="pushdown_dnf-125"><a href="#pushdown_dnf-125"><span class="linenos">125</span></a> <span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-126"><a href="#pushdown_dnf-126"><span class="linenos">126</span></a>
|
||||
</span><span id="pushdown_dnf-127"><a href="#pushdown_dnf-127"><span class="linenos">127</span></a> <span class="k">if</span> <span class="n">predicate_condition</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-128"><a href="#pushdown_dnf-128"><span class="linenos">128</span></a> <span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="pushdown_dnf-129"><a href="#pushdown_dnf-129"><span class="linenos">129</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span><span class="p">(</span><span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">],</span> <span class="n">predicate_condition</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-130"><a href="#pushdown_dnf-130"><span class="linenos">130</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">conditions</span>
|
||||
</span><span id="pushdown_dnf-131"><a href="#pushdown_dnf-131"><span class="linenos">131</span></a> <span class="k">else</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="pushdown_dnf-132"><a href="#pushdown_dnf-132"><span class="linenos">132</span></a> <span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-133"><a href="#pushdown_dnf-133"><span class="linenos">133</span></a>
|
||||
</span><span id="pushdown_dnf-134"><a href="#pushdown_dnf-134"><span class="linenos">134</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="pushdown_dnf-135"><a href="#pushdown_dnf-135"><span class="linenos">135</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">conditions</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-136"><a href="#pushdown_dnf-136"><span class="linenos">136</span></a> <span class="k">continue</span>
|
||||
</span><span id="pushdown_dnf-137"><a href="#pushdown_dnf-137"><span class="linenos">137</span></a>
|
||||
</span><span id="pushdown_dnf-138"><a href="#pushdown_dnf-138"><span class="linenos">138</span></a> <span class="n">predicate</span> <span class="o">=</span> <span class="n">conditions</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
|
||||
</span><span id="pushdown_dnf-139"><a href="#pushdown_dnf-139"><span class="linenos">139</span></a>
|
||||
</span><span id="pushdown_dnf-140"><a href="#pushdown_dnf-140"><span class="linenos">140</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-141"><a href="#pushdown_dnf-141"><span class="linenos">141</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-142"><a href="#pushdown_dnf-142"><span class="linenos">142</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-143"><a href="#pushdown_dnf-143"><span class="linenos">143</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-116"><a href="#pushdown_dnf-116"><span class="linenos">116</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">nodes</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-117"><a href="#pushdown_dnf-117"><span class="linenos">117</span></a> <span class="k">continue</span>
|
||||
</span><span id="pushdown_dnf-118"><a href="#pushdown_dnf-118"><span class="linenos">118</span></a>
|
||||
</span><span id="pushdown_dnf-119"><a href="#pushdown_dnf-119"><span class="linenos">119</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="pushdown_dnf-120"><a href="#pushdown_dnf-120"><span class="linenos">120</span></a>
|
||||
</span><span id="pushdown_dnf-121"><a href="#pushdown_dnf-121"><span class="linenos">121</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">predicate</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-122"><a href="#pushdown_dnf-122"><span class="linenos">122</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">table</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-123"><a href="#pushdown_dnf-123"><span class="linenos">123</span></a> <span class="n">condition</span> <span class="o">=</span> <span class="n">column</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Condition</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-124"><a href="#pushdown_dnf-124"><span class="linenos">124</span></a> <span class="n">predicate_condition</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="pushdown_dnf-125"><a href="#pushdown_dnf-125"><span class="linenos">125</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">and_</span><span class="p">(</span><span class="n">predicate_condition</span><span class="p">,</span> <span class="n">condition</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-126"><a href="#pushdown_dnf-126"><span class="linenos">126</span></a> <span class="k">if</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="pushdown_dnf-127"><a href="#pushdown_dnf-127"><span class="linenos">127</span></a> <span class="k">else</span> <span class="n">condition</span>
|
||||
</span><span id="pushdown_dnf-128"><a href="#pushdown_dnf-128"><span class="linenos">128</span></a> <span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-129"><a href="#pushdown_dnf-129"><span class="linenos">129</span></a>
|
||||
</span><span id="pushdown_dnf-130"><a href="#pushdown_dnf-130"><span class="linenos">130</span></a> <span class="k">if</span> <span class="n">predicate_condition</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-131"><a href="#pushdown_dnf-131"><span class="linenos">131</span></a> <span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span>
|
||||
</span><span id="pushdown_dnf-132"><a href="#pushdown_dnf-132"><span class="linenos">132</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">or_</span><span class="p">(</span><span class="n">conditions</span><span class="p">[</span><span class="n">table</span><span class="p">],</span> <span class="n">predicate_condition</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-133"><a href="#pushdown_dnf-133"><span class="linenos">133</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">conditions</span>
|
||||
</span><span id="pushdown_dnf-134"><a href="#pushdown_dnf-134"><span class="linenos">134</span></a> <span class="k">else</span> <span class="n">predicate_condition</span>
|
||||
</span><span id="pushdown_dnf-135"><a href="#pushdown_dnf-135"><span class="linenos">135</span></a> <span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-136"><a href="#pushdown_dnf-136"><span class="linenos">136</span></a>
|
||||
</span><span id="pushdown_dnf-137"><a href="#pushdown_dnf-137"><span class="linenos">137</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">node</span> <span class="ow">in</span> <span class="n">nodes</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="pushdown_dnf-138"><a href="#pushdown_dnf-138"><span class="linenos">138</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">conditions</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-139"><a href="#pushdown_dnf-139"><span class="linenos">139</span></a> <span class="k">continue</span>
|
||||
</span><span id="pushdown_dnf-140"><a href="#pushdown_dnf-140"><span class="linenos">140</span></a>
|
||||
</span><span id="pushdown_dnf-141"><a href="#pushdown_dnf-141"><span class="linenos">141</span></a> <span class="n">predicate</span> <span class="o">=</span> <span class="n">conditions</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
|
||||
</span><span id="pushdown_dnf-142"><a href="#pushdown_dnf-142"><span class="linenos">142</span></a>
|
||||
</span><span id="pushdown_dnf-143"><a href="#pushdown_dnf-143"><span class="linenos">143</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-144"><a href="#pushdown_dnf-144"><span class="linenos">144</span></a> <span class="n">node</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-145"><a href="#pushdown_dnf-145"><span class="linenos">145</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="pushdown_dnf-146"><a href="#pushdown_dnf-146"><span class="linenos">146</span></a> <span class="n">node</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">replace_aliases</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">predicate</span><span class="p">),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -515,44 +521,44 @@ Additionally, we can't remove predicates from their original form.</p>
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#nodes_for_predicate"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="nodes_for_predicate-146"><a href="#nodes_for_predicate-146"><span class="linenos">146</span></a><span class="k">def</span> <span class="nf">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-147"><a href="#nodes_for_predicate-147"><span class="linenos">147</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="nodes_for_predicate-148"><a href="#nodes_for_predicate-148"><span class="linenos">148</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-149"><a href="#nodes_for_predicate-149"><span class="linenos">149</span></a> <span class="n">where_condition</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-150"><a href="#nodes_for_predicate-150"><span class="linenos">150</span></a>
|
||||
</span><span id="nodes_for_predicate-151"><a href="#nodes_for_predicate-151"><span class="linenos">151</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">tables</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-152"><a href="#nodes_for_predicate-152"><span class="linenos">152</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">source</span> <span class="o">=</span> <span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="ow">or</span> <span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="nodes_for_predicate-149"><a href="#nodes_for_predicate-149"><span class="linenos">149</span></a><span class="k">def</span> <span class="nf">nodes_for_predicate</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">sources</span><span class="p">,</span> <span class="n">scope_ref_count</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-150"><a href="#nodes_for_predicate-150"><span class="linenos">150</span></a> <span class="n">nodes</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="nodes_for_predicate-151"><a href="#nodes_for_predicate-151"><span class="linenos">151</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-152"><a href="#nodes_for_predicate-152"><span class="linenos">152</span></a> <span class="n">where_condition</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-153"><a href="#nodes_for_predicate-153"><span class="linenos">153</span></a>
|
||||
</span><span id="nodes_for_predicate-154"><a href="#nodes_for_predicate-154"><span class="linenos">154</span></a> <span class="c1"># if the predicate is in a where statement we can try to push it down</span>
|
||||
</span><span id="nodes_for_predicate-155"><a href="#nodes_for_predicate-155"><span class="linenos">155</span></a> <span class="c1"># we want to find the root join or from statement</span>
|
||||
</span><span id="nodes_for_predicate-156"><a href="#nodes_for_predicate-156"><span class="linenos">156</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">where_condition</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-157"><a href="#nodes_for_predicate-157"><span class="linenos">157</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-158"><a href="#nodes_for_predicate-158"><span class="linenos">158</span></a>
|
||||
</span><span id="nodes_for_predicate-159"><a href="#nodes_for_predicate-159"><span class="linenos">159</span></a> <span class="c1"># a node can reference a CTE which should be pushed down</span>
|
||||
</span><span id="nodes_for_predicate-160"><a href="#nodes_for_predicate-160"><span class="linenos">160</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-161"><a href="#nodes_for_predicate-161"><span class="linenos">161</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"with"</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-162"><a href="#nodes_for_predicate-162"><span class="linenos">162</span></a> <span class="k">if</span> <span class="n">with_</span> <span class="ow">and</span> <span class="n">with_</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-163"><a href="#nodes_for_predicate-163"><span class="linenos">163</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="nodes_for_predicate-164"><a href="#nodes_for_predicate-164"><span class="linenos">164</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="nodes_for_predicate-165"><a href="#nodes_for_predicate-165"><span class="linenos">165</span></a>
|
||||
</span><span id="nodes_for_predicate-166"><a href="#nodes_for_predicate-166"><span class="linenos">166</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-167"><a href="#nodes_for_predicate-167"><span class="linenos">167</span></a> <span class="k">if</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="o">!=</span> <span class="s2">"RIGHT"</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-168"><a href="#nodes_for_predicate-168"><span class="linenos">168</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="nodes_for_predicate-169"><a href="#nodes_for_predicate-169"><span class="linenos">169</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="nodes_for_predicate-170"><a href="#nodes_for_predicate-170"><span class="linenos">170</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-171"><a href="#nodes_for_predicate-171"><span class="linenos">171</span></a> <span class="c1"># We can't push down window expressions</span>
|
||||
</span><span id="nodes_for_predicate-172"><a href="#nodes_for_predicate-172"><span class="linenos">172</span></a> <span class="n">has_window_expression</span> <span class="o">=</span> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="nodes_for_predicate-173"><a href="#nodes_for_predicate-173"><span class="linenos">173</span></a> <span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-174"><a href="#nodes_for_predicate-174"><span class="linenos">174</span></a> <span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-175"><a href="#nodes_for_predicate-175"><span class="linenos">175</span></a> <span class="c1"># we can't push down predicates to select statements if they are referenced in</span>
|
||||
</span><span id="nodes_for_predicate-176"><a href="#nodes_for_predicate-176"><span class="linenos">176</span></a> <span class="c1"># multiple places.</span>
|
||||
</span><span id="nodes_for_predicate-177"><a href="#nodes_for_predicate-177"><span class="linenos">177</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="nodes_for_predicate-178"><a href="#nodes_for_predicate-178"><span class="linenos">178</span></a> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"group"</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-179"><a href="#nodes_for_predicate-179"><span class="linenos">179</span></a> <span class="ow">and</span> <span class="n">scope_ref_count</span><span class="p">[</span><span class="nb">id</span><span class="p">(</span><span class="n">source</span><span class="p">)]</span> <span class="o"><</span> <span class="mi">2</span>
|
||||
</span><span id="nodes_for_predicate-180"><a href="#nodes_for_predicate-180"><span class="linenos">180</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">has_window_expression</span>
|
||||
</span><span id="nodes_for_predicate-181"><a href="#nodes_for_predicate-181"><span class="linenos">181</span></a> <span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-182"><a href="#nodes_for_predicate-182"><span class="linenos">182</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="nodes_for_predicate-183"><a href="#nodes_for_predicate-183"><span class="linenos">183</span></a> <span class="k">return</span> <span class="n">nodes</span>
|
||||
</span><span id="nodes_for_predicate-154"><a href="#nodes_for_predicate-154"><span class="linenos">154</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">tables</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-155"><a href="#nodes_for_predicate-155"><span class="linenos">155</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">source</span> <span class="o">=</span> <span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="ow">or</span> <span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-156"><a href="#nodes_for_predicate-156"><span class="linenos">156</span></a>
|
||||
</span><span id="nodes_for_predicate-157"><a href="#nodes_for_predicate-157"><span class="linenos">157</span></a> <span class="c1"># if the predicate is in a where statement we can try to push it down</span>
|
||||
</span><span id="nodes_for_predicate-158"><a href="#nodes_for_predicate-158"><span class="linenos">158</span></a> <span class="c1"># we want to find the root join or from statement</span>
|
||||
</span><span id="nodes_for_predicate-159"><a href="#nodes_for_predicate-159"><span class="linenos">159</span></a> <span class="k">if</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">where_condition</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-160"><a href="#nodes_for_predicate-160"><span class="linenos">160</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-161"><a href="#nodes_for_predicate-161"><span class="linenos">161</span></a>
|
||||
</span><span id="nodes_for_predicate-162"><a href="#nodes_for_predicate-162"><span class="linenos">162</span></a> <span class="c1"># a node can reference a CTE which should be pushed down</span>
|
||||
</span><span id="nodes_for_predicate-163"><a href="#nodes_for_predicate-163"><span class="linenos">163</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-164"><a href="#nodes_for_predicate-164"><span class="linenos">164</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">parent</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"with"</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-165"><a href="#nodes_for_predicate-165"><span class="linenos">165</span></a> <span class="k">if</span> <span class="n">with_</span> <span class="ow">and</span> <span class="n">with_</span><span class="o">.</span><span class="n">recursive</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-166"><a href="#nodes_for_predicate-166"><span class="linenos">166</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="nodes_for_predicate-167"><a href="#nodes_for_predicate-167"><span class="linenos">167</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="nodes_for_predicate-168"><a href="#nodes_for_predicate-168"><span class="linenos">168</span></a>
|
||||
</span><span id="nodes_for_predicate-169"><a href="#nodes_for_predicate-169"><span class="linenos">169</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-170"><a href="#nodes_for_predicate-170"><span class="linenos">170</span></a> <span class="k">if</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">side</span> <span class="o">!=</span> <span class="s2">"RIGHT"</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-171"><a href="#nodes_for_predicate-171"><span class="linenos">171</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="nodes_for_predicate-172"><a href="#nodes_for_predicate-172"><span class="linenos">172</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="nodes_for_predicate-173"><a href="#nodes_for_predicate-173"><span class="linenos">173</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">tables</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-174"><a href="#nodes_for_predicate-174"><span class="linenos">174</span></a> <span class="c1"># We can't push down window expressions</span>
|
||||
</span><span id="nodes_for_predicate-175"><a href="#nodes_for_predicate-175"><span class="linenos">175</span></a> <span class="n">has_window_expression</span> <span class="o">=</span> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="nodes_for_predicate-176"><a href="#nodes_for_predicate-176"><span class="linenos">176</span></a> <span class="n">select</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">node</span><span class="o">.</span><span class="n">selects</span> <span class="k">if</span> <span class="n">select</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Window</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-177"><a href="#nodes_for_predicate-177"><span class="linenos">177</span></a> <span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-178"><a href="#nodes_for_predicate-178"><span class="linenos">178</span></a> <span class="c1"># we can't push down predicates to select statements if they are referenced in</span>
|
||||
</span><span id="nodes_for_predicate-179"><a href="#nodes_for_predicate-179"><span class="linenos">179</span></a> <span class="c1"># multiple places.</span>
|
||||
</span><span id="nodes_for_predicate-180"><a href="#nodes_for_predicate-180"><span class="linenos">180</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="nodes_for_predicate-181"><a href="#nodes_for_predicate-181"><span class="linenos">181</span></a> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"group"</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-182"><a href="#nodes_for_predicate-182"><span class="linenos">182</span></a> <span class="ow">and</span> <span class="n">scope_ref_count</span><span class="p">[</span><span class="nb">id</span><span class="p">(</span><span class="n">source</span><span class="p">)]</span> <span class="o"><</span> <span class="mi">2</span>
|
||||
</span><span id="nodes_for_predicate-183"><a href="#nodes_for_predicate-183"><span class="linenos">183</span></a> <span class="ow">and</span> <span class="ow">not</span> <span class="n">has_window_expression</span>
|
||||
</span><span id="nodes_for_predicate-184"><a href="#nodes_for_predicate-184"><span class="linenos">184</span></a> <span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-185"><a href="#nodes_for_predicate-185"><span class="linenos">185</span></a> <span class="n">nodes</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">node</span>
|
||||
</span><span id="nodes_for_predicate-186"><a href="#nodes_for_predicate-186"><span class="linenos">186</span></a> <span class="k">return</span> <span class="n">nodes</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -570,21 +576,21 @@ Additionally, we can't remove predicates from their original form.</p>
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#replace_aliases"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="replace_aliases-186"><a href="#replace_aliases-186"><span class="linenos">186</span></a><span class="k">def</span> <span class="nf">replace_aliases</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">predicate</span><span class="p">):</span>
|
||||
</span><span id="replace_aliases-187"><a href="#replace_aliases-187"><span class="linenos">187</span></a> <span class="n">aliases</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="replace_aliases-188"><a href="#replace_aliases-188"><span class="linenos">188</span></a>
|
||||
</span><span id="replace_aliases-189"><a href="#replace_aliases-189"><span class="linenos">189</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="replace_aliases-190"><a href="#replace_aliases-190"><span class="linenos">190</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="replace_aliases-191"><a href="#replace_aliases-191"><span class="linenos">191</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="replace_aliases-192"><a href="#replace_aliases-192"><span class="linenos">192</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="replace_aliases-193"><a href="#replace_aliases-193"><span class="linenos">193</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span>
|
||||
</span><span id="replace_aliases-194"><a href="#replace_aliases-194"><span class="linenos">194</span></a>
|
||||
</span><span id="replace_aliases-195"><a href="#replace_aliases-195"><span class="linenos">195</span></a> <span class="k">def</span> <span class="nf">_replace_alias</span><span class="p">(</span><span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="replace_aliases-196"><a href="#replace_aliases-196"><span class="linenos">196</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="ow">and</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">aliases</span><span class="p">:</span>
|
||||
</span><span id="replace_aliases-197"><a href="#replace_aliases-197"><span class="linenos">197</span></a> <span class="k">return</span> <span class="n">aliases</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
</span><span id="replace_aliases-198"><a href="#replace_aliases-198"><span class="linenos">198</span></a> <span class="k">return</span> <span class="n">column</span>
|
||||
</span><span id="replace_aliases-199"><a href="#replace_aliases-199"><span class="linenos">199</span></a>
|
||||
</span><span id="replace_aliases-200"><a href="#replace_aliases-200"><span class="linenos">200</span></a> <span class="k">return</span> <span class="n">predicate</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_replace_alias</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="replace_aliases-189"><a href="#replace_aliases-189"><span class="linenos">189</span></a><span class="k">def</span> <span class="nf">replace_aliases</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">predicate</span><span class="p">):</span>
|
||||
</span><span id="replace_aliases-190"><a href="#replace_aliases-190"><span class="linenos">190</span></a> <span class="n">aliases</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="replace_aliases-191"><a href="#replace_aliases-191"><span class="linenos">191</span></a>
|
||||
</span><span id="replace_aliases-192"><a href="#replace_aliases-192"><span class="linenos">192</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">source</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="replace_aliases-193"><a href="#replace_aliases-193"><span class="linenos">193</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">select</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
</span><span id="replace_aliases-194"><a href="#replace_aliases-194"><span class="linenos">194</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">alias</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="replace_aliases-195"><a href="#replace_aliases-195"><span class="linenos">195</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="replace_aliases-196"><a href="#replace_aliases-196"><span class="linenos">196</span></a> <span class="n">aliases</span><span class="p">[</span><span class="n">select</span><span class="o">.</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">select</span>
|
||||
</span><span id="replace_aliases-197"><a href="#replace_aliases-197"><span class="linenos">197</span></a>
|
||||
</span><span id="replace_aliases-198"><a href="#replace_aliases-198"><span class="linenos">198</span></a> <span class="k">def</span> <span class="nf">_replace_alias</span><span class="p">(</span><span class="n">column</span><span class="p">):</span>
|
||||
</span><span id="replace_aliases-199"><a href="#replace_aliases-199"><span class="linenos">199</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Column</span><span class="p">)</span> <span class="ow">and</span> <span class="n">column</span><span class="o">.</span><span class="n">name</span> <span class="ow">in</span> <span class="n">aliases</span><span class="p">:</span>
|
||||
</span><span id="replace_aliases-200"><a href="#replace_aliases-200"><span class="linenos">200</span></a> <span class="k">return</span> <span class="n">aliases</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
</span><span id="replace_aliases-201"><a href="#replace_aliases-201"><span class="linenos">201</span></a> <span class="k">return</span> <span class="n">column</span>
|
||||
</span><span id="replace_aliases-202"><a href="#replace_aliases-202"><span class="linenos">202</span></a>
|
||||
</span><span id="replace_aliases-203"><a href="#replace_aliases-203"><span class="linenos">203</span></a> <span class="k">return</span> <span class="n">predicate</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_replace_alias</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -112,12 +112,12 @@
|
|||
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="n">left</span><span class="p">,</span> <span class="n">right</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">union_scopes</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">left</span><span class="p">]</span> <span class="o">=</span> <span class="n">parent_selections</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="k">if</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">right</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="k">if</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="n">parent_selections</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="k">elif</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">left</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="k">elif</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">right</span><span class="o">.</span><span class="n">selects</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">select</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">left</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">select</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="k">if</span> <span class="n">SELECT_ALL</span> <span class="ow">in</span> <span class="n">parent_selections</span> <span class="ow">or</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">in</span> <span class="n">parent_selections</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="p">]</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a>
|
||||
|
|
@ -154,7 +154,7 @@
|
|||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="n">removed</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="n">star</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">for</span> <span class="n">selection</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="k">for</span> <span class="n">selection</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">selection</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="k">if</span> <span class="n">SELECT_ALL</span> <span class="ow">in</span> <span class="n">parent_selections</span> <span class="ow">or</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">parent_selections</span> <span class="ow">or</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">order_refs</span><span class="p">:</span>
|
||||
|
|
@ -263,12 +263,12 @@
|
|||
</span><span id="pushdown_projections-49"><a href="#pushdown_projections-49"><span class="linenos">49</span></a> <span class="n">left</span><span class="p">,</span> <span class="n">right</span> <span class="o">=</span> <span class="n">scope</span><span class="o">.</span><span class="n">union_scopes</span>
|
||||
</span><span id="pushdown_projections-50"><a href="#pushdown_projections-50"><span class="linenos">50</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">left</span><span class="p">]</span> <span class="o">=</span> <span class="n">parent_selections</span>
|
||||
</span><span id="pushdown_projections-51"><a href="#pushdown_projections-51"><span class="linenos">51</span></a>
|
||||
</span><span id="pushdown_projections-52"><a href="#pushdown_projections-52"><span class="linenos">52</span></a> <span class="k">if</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">right</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
|
||||
</span><span id="pushdown_projections-52"><a href="#pushdown_projections-52"><span class="linenos">52</span></a> <span class="k">if</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
|
||||
</span><span id="pushdown_projections-53"><a href="#pushdown_projections-53"><span class="linenos">53</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="n">parent_selections</span>
|
||||
</span><span id="pushdown_projections-54"><a href="#pushdown_projections-54"><span class="linenos">54</span></a> <span class="k">elif</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">left</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
|
||||
</span><span id="pushdown_projections-54"><a href="#pushdown_projections-54"><span class="linenos">54</span></a> <span class="k">elif</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">select</span><span class="o">.</span><span class="n">is_star</span> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
|
||||
</span><span id="pushdown_projections-55"><a href="#pushdown_projections-55"><span class="linenos">55</span></a> <span class="n">referenced_columns</span><span class="p">[</span><span class="n">right</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span>
|
||||
</span><span id="pushdown_projections-56"><a href="#pushdown_projections-56"><span class="linenos">56</span></a> <span class="n">right</span><span class="o">.</span><span class="n">selects</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="pushdown_projections-57"><a href="#pushdown_projections-57"><span class="linenos">57</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">select</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">left</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span>
|
||||
</span><span id="pushdown_projections-56"><a href="#pushdown_projections-56"><span class="linenos">56</span></a> <span class="n">right</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="pushdown_projections-57"><a href="#pushdown_projections-57"><span class="linenos">57</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">select</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">left</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">)</span>
|
||||
</span><span id="pushdown_projections-58"><a href="#pushdown_projections-58"><span class="linenos">58</span></a> <span class="k">if</span> <span class="n">SELECT_ALL</span> <span class="ow">in</span> <span class="n">parent_selections</span> <span class="ow">or</span> <span class="n">select</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="ow">in</span> <span class="n">parent_selections</span>
|
||||
</span><span id="pushdown_projections-59"><a href="#pushdown_projections-59"><span class="linenos">59</span></a> <span class="p">]</span>
|
||||
</span><span id="pushdown_projections-60"><a href="#pushdown_projections-60"><span class="linenos">60</span></a>
|
||||
|
|
|
|||
|
|
@ -280,13 +280,13 @@
|
|||
</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="k">if</span> <span class="n">table</span> <span class="ow">and</span> <span class="p">(</span><span class="ow">not</span> <span class="n">alias_expr</span> <span class="ow">or</span> <span class="n">double_agg</span><span class="p">):</span>
|
||||
</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="n">column</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"table"</span><span class="p">,</span> <span class="n">table</span><span class="p">)</span>
|
||||
</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="k">elif</span> <span class="ow">not</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="ow">and</span> <span class="n">alias_expr</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">double_agg</span><span class="p">:</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">alias_expr</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Literal</span><span class="p">):</span>
|
||||
</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">alias_expr</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Literal</span><span class="p">)</span> <span class="ow">and</span> <span class="p">(</span><span class="n">literal_index</span> <span class="ow">or</span> <span class="n">resolve_table</span><span class="p">):</span>
|
||||
</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="k">if</span> <span class="n">literal_index</span><span class="p">:</span>
|
||||
</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="n">column</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Literal</span><span class="o">.</span><span class="n">number</span><span class="p">(</span><span class="n">i</span><span class="p">))</span>
|
||||
</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="n">column</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">alias_expr</span><span class="o">.</span><span class="n">copy</span><span class="p">())</span>
|
||||
</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">projection</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
|
||||
</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">projection</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">):</span>
|
||||
</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="n">replace_columns</span><span class="p">(</span><span class="n">projection</span><span class="p">)</span>
|
||||
</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a>
|
||||
</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">projection</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">):</span>
|
||||
|
|
@ -327,7 +327,7 @@
|
|||
</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a> <span class="n">ordered</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"this"</span><span class="p">,</span> <span class="n">new_expression</span><span class="p">)</span>
|
||||
</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a>
|
||||
</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a> <span class="k">if</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"group"</span><span class="p">):</span>
|
||||
</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a> <span class="n">selects</span> <span class="o">=</span> <span class="p">{</span><span class="n">s</span><span class="o">.</span><span class="n">this</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">s</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">)</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selects</span><span class="p">}</span>
|
||||
</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a> <span class="n">selects</span> <span class="o">=</span> <span class="p">{</span><span class="n">s</span><span class="o">.</span><span class="n">this</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">column</span><span class="p">(</span><span class="n">s</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">)</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">}</span>
|
||||
</span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a>
|
||||
</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="k">for</span> <span class="n">ordered</span> <span class="ow">in</span> <span class="n">ordereds</span><span class="p">:</span>
|
||||
</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a> <span class="n">ordered</span> <span class="o">=</span> <span class="n">ordered</span><span class="o">.</span><span class="n">this</span>
|
||||
|
|
@ -358,7 +358,7 @@
|
|||
</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a>
|
||||
</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a><span class="k">def</span> <span class="nf">_select_by_pos</span><span class="p">(</span><span class="n">scope</span><span class="p">:</span> <span class="n">Scope</span><span class="p">,</span> <span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Literal</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">:</span>
|
||||
</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a> <span class="k">return</span> <span class="n">scope</span><span class="o">.</span><span class="n">selects</span><span class="p">[</span><span class="nb">int</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">assert_is</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">)</span>
|
||||
</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a> <span class="k">return</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">[</span><span class="nb">int</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="o">-</span> <span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">assert_is</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Alias</span><span class="p">)</span>
|
||||
</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a> <span class="k">except</span> <span class="ne">IndexError</span><span class="p">:</span>
|
||||
</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown output column: </span><span class="si">{</span><span class="n">node</span><span class="o">.</span><span class="n">name</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a>
|
||||
|
|
@ -435,7 +435,7 @@
|
|||
</span><span id="L-347"><a href="#L-347"><span class="linenos">347</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">pivot_output_columns</span><span class="p">:</span>
|
||||
</span><span id="L-348"><a href="#L-348"><span class="linenos">348</span></a> <span class="n">pivot_output_columns</span> <span class="o">=</span> <span class="p">[</span><span class="n">col</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">pivot</span><span class="o">.</span><span class="n">expressions</span><span class="p">]</span>
|
||||
</span><span id="L-349"><a href="#L-349"><span class="linenos">349</span></a>
|
||||
</span><span id="L-350"><a href="#L-350"><span class="linenos">350</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-350"><a href="#L-350"><span class="linenos">350</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">:</span>
|
||||
</span><span id="L-351"><a href="#L-351"><span class="linenos">351</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Star</span><span class="p">):</span>
|
||||
</span><span id="L-352"><a href="#L-352"><span class="linenos">352</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span>
|
||||
</span><span id="L-353"><a href="#L-353"><span class="linenos">353</span></a> <span class="n">_add_except_columns</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">tables</span><span class="p">,</span> <span class="n">except_columns</span><span class="p">)</span>
|
||||
|
|
@ -534,7 +534,7 @@
|
|||
</span><span id="L-446"><a href="#L-446"><span class="linenos">446</span></a> <span class="n">new_selections</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-447"><a href="#L-447"><span class="linenos">447</span></a>
|
||||
</span><span id="L-448"><a href="#L-448"><span class="linenos">448</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="p">(</span><span class="n">selection</span><span class="p">,</span> <span class="n">aliased_column</span><span class="p">)</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span>
|
||||
</span><span id="L-449"><a href="#L-449"><span class="linenos">449</span></a> <span class="n">itertools</span><span class="o">.</span><span class="n">zip_longest</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">selects</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">outer_column_list</span><span class="p">)</span>
|
||||
</span><span id="L-449"><a href="#L-449"><span class="linenos">449</span></a> <span class="n">itertools</span><span class="o">.</span><span class="n">zip_longest</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">selects</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">outer_column_list</span><span class="p">)</span>
|
||||
</span><span id="L-450"><a href="#L-450"><span class="linenos">450</span></a> <span class="p">):</span>
|
||||
</span><span id="L-451"><a href="#L-451"><span class="linenos">451</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">selection</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
|
||||
</span><span id="L-452"><a href="#L-452"><span class="linenos">452</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">selection</span><span class="o">.</span><span class="n">output_name</span><span class="p">:</span>
|
||||
|
|
|
|||
|
|
@ -56,111 +56,104 @@
|
|||
|
||||
<label class="view-source-button" for="mod-qualify_tables-view-source"><span>View Source</span></label>
|
||||
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="kn">import</span> <span class="nn">itertools</span>
|
||||
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="kn">import</span> <span class="nn">typing</span> <span class="k">as</span> <span class="nn">t</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">alias</span><span class="p">,</span> <span class="n">exp</span>
|
||||
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="kn">from</span> <span class="nn">sqlglot._typing</span> <span class="kn">import</span> <span class="n">E</span>
|
||||
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot.helper</span> <span class="kn">import</span> <span class="n">csv_reader</span><span class="p">,</span> <span class="n">name_sequence</span>
|
||||
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="n">Scope</span><span class="p">,</span> <span class="n">traverse_scope</span>
|
||||
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span> <span class="nn">sqlglot.schema</span> <span class="kn">import</span> <span class="n">Schema</span>
|
||||
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a>
|
||||
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a>
|
||||
</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="k">def</span> <span class="nf">qualify_tables</span><span class="p">(</span>
|
||||
</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span>
|
||||
</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Additionally, this</span>
|
||||
</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a><span class="sd"> replaces "join constructs" (*) by equivalent SELECT * subqueries.</span>
|
||||
</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a><span class="sd"> Examples:</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT 1 FROM tbl")</span>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a><span class="sd"> >>> qualify_tables(expression, db="db").sql()</span>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="sd"> 'SELECT 1 FROM db.tbl AS tbl'</span>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a><span class="sd"> >>></span>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT * FROM (tbl1 JOIN tbl2 ON id1 = id2)")</span>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a><span class="sd"> >>> qualify_tables(expression).sql()</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="sd"> 'SELECT * FROM (SELECT * FROM tbl1 AS tbl1 JOIN tbl2 AS tbl2 ON id1 = id2) AS _q_0'</span>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a><span class="sd"> expression: Expression to qualify</span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a><span class="sd"> db: Database name</span>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a><span class="sd"> catalog: Catalog name</span>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a><span class="sd"> schema: A schema to populate</span>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a><span class="sd"> The qualified expression.</span>
|
||||
</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a>
|
||||
</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="sd"> (*) See section 7.2.1.2 in https://www.postgresql.org/docs/current/queries-table-expressions.html</span>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">"_q_"</span><span class="p">)</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="c1"># Expand join construct</span>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">"*"</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">):</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"db"</span><span class="p">):</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"db"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">))</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">):</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">))</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">alias</span><span class="p">(</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="n">source</span><span class="p">,</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="p">)</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="p">)</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="p">)</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="k">if</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="n">source</span><span class="p">,</span> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)}</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="p">)</span>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="p">)</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"this"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
|
||||
</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"columns"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">"_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">"</span><span class="p">))</span>
|
||||
</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
|
||||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="kn">import</span> <span class="nn">itertools</span>
|
||||
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="kn">import</span> <span class="nn">typing</span> <span class="k">as</span> <span class="nn">t</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">alias</span><span class="p">,</span> <span class="n">exp</span>
|
||||
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="kn">from</span> <span class="nn">sqlglot._typing</span> <span class="kn">import</span> <span class="n">E</span>
|
||||
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot.helper</span> <span class="kn">import</span> <span class="n">csv_reader</span><span class="p">,</span> <span class="n">name_sequence</span>
|
||||
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="n">Scope</span><span class="p">,</span> <span class="n">traverse_scope</span>
|
||||
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span> <span class="nn">sqlglot.schema</span> <span class="kn">import</span> <span class="n">Schema</span>
|
||||
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a>
|
||||
</span><span id="L-10"><a href="#L-10"><span class="linenos">10</span></a>
|
||||
</span><span id="L-11"><a href="#L-11"><span class="linenos">11</span></a><span class="k">def</span> <span class="nf">qualify_tables</span><span class="p">(</span>
|
||||
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span>
|
||||
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span>
|
||||
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span>
|
||||
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a><span class="sd"> Examples:</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos">22</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos">23</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT 1 FROM tbl")</span>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos">24</span></a><span class="sd"> >>> qualify_tables(expression, db="db").sql()</span>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos">25</span></a><span class="sd"> 'SELECT 1 FROM db.tbl AS tbl'</span>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos">26</span></a><span class="sd"> >>></span>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos">27</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT 1 FROM (t1 JOIN t2) AS t")</span>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos">28</span></a><span class="sd"> >>> qualify_tables(expression).sql()</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos">29</span></a><span class="sd"> 'SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t'</span>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos">30</span></a>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos">31</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos">32</span></a><span class="sd"> expression: Expression to qualify</span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos">33</span></a><span class="sd"> db: Database name</span>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos">34</span></a><span class="sd"> catalog: Catalog name</span>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos">35</span></a><span class="sd"> schema: A schema to populate</span>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos">36</span></a>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos">37</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos">38</span></a><span class="sd"> The qualified expression.</span>
|
||||
</span><span id="L-39"><a href="#L-39"><span class="linenos">39</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-40"><a href="#L-40"><span class="linenos">40</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">"_q_"</span><span class="p">)</span>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos">41</span></a>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos">42</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos">43</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos">44</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos">45</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos">46</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos">47</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos">48</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">"*"</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">):</span>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos">63</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"db"</span><span class="p">):</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos">64</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"db"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">))</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos">65</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">):</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos">66</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">))</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos">67</span></a>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos">68</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos">69</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos">70</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos">71</span></a>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos">72</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos">73</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos">74</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a> <span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos">76</span></a> <span class="p">)</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a> <span class="k">if</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos">83</span></a> <span class="n">source</span><span class="p">,</span> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)}</span>
|
||||
</span><span id="L-84"><a href="#L-84"><span class="linenos">84</span></a> <span class="p">)</span>
|
||||
</span><span id="L-85"><a href="#L-85"><span class="linenos">85</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
|
||||
</span><span id="L-86"><a href="#L-86"><span class="linenos">86</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-87"><a href="#L-87"><span class="linenos">87</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
|
||||
</span><span id="L-88"><a href="#L-88"><span class="linenos">88</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
|
||||
</span><span id="L-89"><a href="#L-89"><span class="linenos">89</span></a> <span class="p">)</span>
|
||||
</span><span id="L-90"><a href="#L-90"><span class="linenos">90</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
|
||||
</span><span id="L-91"><a href="#L-91"><span class="linenos">91</span></a>
|
||||
</span><span id="L-92"><a href="#L-92"><span class="linenos">92</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="L-93"><a href="#L-93"><span class="linenos">93</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"this"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos">94</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos">95</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos">96</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"columns"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">"_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">"</span><span class="p">))</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos">97</span></a>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos">98</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -176,106 +169,99 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#qualify_tables"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="qualify_tables-12"><a href="#qualify_tables-12"><span class="linenos"> 12</span></a><span class="k">def</span> <span class="nf">qualify_tables</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-13"><a href="#qualify_tables-13"><span class="linenos"> 13</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-14"><a href="#qualify_tables-14"><span class="linenos"> 14</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-15"><a href="#qualify_tables-15"><span class="linenos"> 15</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-16"><a href="#qualify_tables-16"><span class="linenos"> 16</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-17"><a href="#qualify_tables-17"><span class="linenos"> 17</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-18"><a href="#qualify_tables-18"><span class="linenos"> 18</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="qualify_tables-19"><a href="#qualify_tables-19"><span class="linenos"> 19</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Additionally, this</span>
|
||||
</span><span id="qualify_tables-20"><a href="#qualify_tables-20"><span class="linenos"> 20</span></a><span class="sd"> replaces "join constructs" (*) by equivalent SELECT * subqueries.</span>
|
||||
</span><span id="qualify_tables-21"><a href="#qualify_tables-21"><span class="linenos"> 21</span></a>
|
||||
</span><span id="qualify_tables-22"><a href="#qualify_tables-22"><span class="linenos"> 22</span></a><span class="sd"> Examples:</span>
|
||||
</span><span id="qualify_tables-23"><a href="#qualify_tables-23"><span class="linenos"> 23</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="qualify_tables-24"><a href="#qualify_tables-24"><span class="linenos"> 24</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT 1 FROM tbl")</span>
|
||||
</span><span id="qualify_tables-25"><a href="#qualify_tables-25"><span class="linenos"> 25</span></a><span class="sd"> >>> qualify_tables(expression, db="db").sql()</span>
|
||||
</span><span id="qualify_tables-26"><a href="#qualify_tables-26"><span class="linenos"> 26</span></a><span class="sd"> 'SELECT 1 FROM db.tbl AS tbl'</span>
|
||||
</span><span id="qualify_tables-27"><a href="#qualify_tables-27"><span class="linenos"> 27</span></a><span class="sd"> >>></span>
|
||||
</span><span id="qualify_tables-28"><a href="#qualify_tables-28"><span class="linenos"> 28</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT * FROM (tbl1 JOIN tbl2 ON id1 = id2)")</span>
|
||||
</span><span id="qualify_tables-29"><a href="#qualify_tables-29"><span class="linenos"> 29</span></a><span class="sd"> >>> qualify_tables(expression).sql()</span>
|
||||
</span><span id="qualify_tables-30"><a href="#qualify_tables-30"><span class="linenos"> 30</span></a><span class="sd"> 'SELECT * FROM (SELECT * FROM tbl1 AS tbl1 JOIN tbl2 AS tbl2 ON id1 = id2) AS _q_0'</span>
|
||||
</span><span id="qualify_tables-31"><a href="#qualify_tables-31"><span class="linenos"> 31</span></a>
|
||||
</span><span id="qualify_tables-32"><a href="#qualify_tables-32"><span class="linenos"> 32</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="qualify_tables-33"><a href="#qualify_tables-33"><span class="linenos"> 33</span></a><span class="sd"> expression: Expression to qualify</span>
|
||||
</span><span id="qualify_tables-34"><a href="#qualify_tables-34"><span class="linenos"> 34</span></a><span class="sd"> db: Database name</span>
|
||||
</span><span id="qualify_tables-35"><a href="#qualify_tables-35"><span class="linenos"> 35</span></a><span class="sd"> catalog: Catalog name</span>
|
||||
</span><span id="qualify_tables-36"><a href="#qualify_tables-36"><span class="linenos"> 36</span></a><span class="sd"> schema: A schema to populate</span>
|
||||
</span><span id="qualify_tables-37"><a href="#qualify_tables-37"><span class="linenos"> 37</span></a>
|
||||
</span><span id="qualify_tables-38"><a href="#qualify_tables-38"><span class="linenos"> 38</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="qualify_tables-39"><a href="#qualify_tables-39"><span class="linenos"> 39</span></a><span class="sd"> The qualified expression.</span>
|
||||
</span><span id="qualify_tables-40"><a href="#qualify_tables-40"><span class="linenos"> 40</span></a>
|
||||
</span><span id="qualify_tables-41"><a href="#qualify_tables-41"><span class="linenos"> 41</span></a><span class="sd"> (*) See section 7.2.1.2 in https://www.postgresql.org/docs/current/queries-table-expressions.html</span>
|
||||
</span><span id="qualify_tables-42"><a href="#qualify_tables-42"><span class="linenos"> 42</span></a><span class="sd"> """</span>
|
||||
</span><span id="qualify_tables-43"><a href="#qualify_tables-43"><span class="linenos"> 43</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">"_q_"</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-44"><a href="#qualify_tables-44"><span class="linenos"> 44</span></a>
|
||||
</span><span id="qualify_tables-45"><a href="#qualify_tables-45"><span class="linenos"> 45</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-46"><a href="#qualify_tables-46"><span class="linenos"> 46</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-47"><a href="#qualify_tables-47"><span class="linenos"> 47</span></a> <span class="c1"># Expand join construct</span>
|
||||
</span><span id="qualify_tables-48"><a href="#qualify_tables-48"><span class="linenos"> 48</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-49"><a href="#qualify_tables-49"><span class="linenos"> 49</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
|
||||
</span><span id="qualify_tables-50"><a href="#qualify_tables-50"><span class="linenos"> 50</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-51"><a href="#qualify_tables-51"><span class="linenos"> 51</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">"*"</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
|
||||
</span><span id="qualify_tables-52"><a href="#qualify_tables-52"><span class="linenos"> 52</span></a>
|
||||
</span><span id="qualify_tables-53"><a href="#qualify_tables-53"><span class="linenos"> 53</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-54"><a href="#qualify_tables-54"><span class="linenos"> 54</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
|
||||
</span><span id="qualify_tables-55"><a href="#qualify_tables-55"><span class="linenos"> 55</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
|
||||
</span><span id="qualify_tables-56"><a href="#qualify_tables-56"><span class="linenos"> 56</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-57"><a href="#qualify_tables-57"><span class="linenos"> 57</span></a>
|
||||
</span><span id="qualify_tables-58"><a href="#qualify_tables-58"><span class="linenos"> 58</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-59"><a href="#qualify_tables-59"><span class="linenos"> 59</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-60"><a href="#qualify_tables-60"><span class="linenos"> 60</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
|
||||
</span><span id="qualify_tables-61"><a href="#qualify_tables-61"><span class="linenos"> 61</span></a>
|
||||
</span><span id="qualify_tables-62"><a href="#qualify_tables-62"><span class="linenos"> 62</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="qualify_tables-63"><a href="#qualify_tables-63"><span class="linenos"> 63</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-64"><a href="#qualify_tables-64"><span class="linenos"> 64</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-65"><a href="#qualify_tables-65"><span class="linenos"> 65</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"db"</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-66"><a href="#qualify_tables-66"><span class="linenos"> 66</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"db"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">))</span>
|
||||
</span><span id="qualify_tables-67"><a href="#qualify_tables-67"><span class="linenos"> 67</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-68"><a href="#qualify_tables-68"><span class="linenos"> 68</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">))</span>
|
||||
</span><span id="qualify_tables-69"><a href="#qualify_tables-69"><span class="linenos"> 69</span></a>
|
||||
</span><span id="qualify_tables-70"><a href="#qualify_tables-70"><span class="linenos"> 70</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-71"><a href="#qualify_tables-71"><span class="linenos"> 71</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-72"><a href="#qualify_tables-72"><span class="linenos"> 72</span></a> <span class="n">alias</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-73"><a href="#qualify_tables-73"><span class="linenos"> 73</span></a> <span class="n">source</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-74"><a href="#qualify_tables-74"><span class="linenos"> 74</span></a> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span>
|
||||
</span><span id="qualify_tables-75"><a href="#qualify_tables-75"><span class="linenos"> 75</span></a> <span class="n">copy</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-76"><a href="#qualify_tables-76"><span class="linenos"> 76</span></a> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-77"><a href="#qualify_tables-77"><span class="linenos"> 77</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-78"><a href="#qualify_tables-78"><span class="linenos"> 78</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-79"><a href="#qualify_tables-79"><span class="linenos"> 79</span></a>
|
||||
</span><span id="qualify_tables-80"><a href="#qualify_tables-80"><span class="linenos"> 80</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-81"><a href="#qualify_tables-81"><span class="linenos"> 81</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-82"><a href="#qualify_tables-82"><span class="linenos"> 82</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-83"><a href="#qualify_tables-83"><span class="linenos"> 83</span></a> <span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="qualify_tables-84"><a href="#qualify_tables-84"><span class="linenos"> 84</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-85"><a href="#qualify_tables-85"><span class="linenos"> 85</span></a>
|
||||
</span><span id="qualify_tables-86"><a href="#qualify_tables-86"><span class="linenos"> 86</span></a> <span class="k">if</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-87"><a href="#qualify_tables-87"><span class="linenos"> 87</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-88"><a href="#qualify_tables-88"><span class="linenos"> 88</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-89"><a href="#qualify_tables-89"><span class="linenos"> 89</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-90"><a href="#qualify_tables-90"><span class="linenos"> 90</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-91"><a href="#qualify_tables-91"><span class="linenos"> 91</span></a> <span class="n">source</span><span class="p">,</span> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)}</span>
|
||||
</span><span id="qualify_tables-92"><a href="#qualify_tables-92"><span class="linenos"> 92</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-93"><a href="#qualify_tables-93"><span class="linenos"> 93</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-94"><a href="#qualify_tables-94"><span class="linenos"> 94</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="qualify_tables-95"><a href="#qualify_tables-95"><span class="linenos"> 95</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-96"><a href="#qualify_tables-96"><span class="linenos"> 96</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
|
||||
</span><span id="qualify_tables-97"><a href="#qualify_tables-97"><span class="linenos"> 97</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-98"><a href="#qualify_tables-98"><span class="linenos"> 98</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-99"><a href="#qualify_tables-99"><span class="linenos"> 99</span></a>
|
||||
</span><span id="qualify_tables-100"><a href="#qualify_tables-100"><span class="linenos">100</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-101"><a href="#qualify_tables-101"><span class="linenos">101</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"this"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="qualify_tables-102"><a href="#qualify_tables-102"><span class="linenos">102</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-103"><a href="#qualify_tables-103"><span class="linenos">103</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-104"><a href="#qualify_tables-104"><span class="linenos">104</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"columns"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">"_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">"</span><span class="p">))</span>
|
||||
</span><span id="qualify_tables-105"><a href="#qualify_tables-105"><span class="linenos">105</span></a>
|
||||
</span><span id="qualify_tables-106"><a href="#qualify_tables-106"><span class="linenos">106</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="qualify_tables-12"><a href="#qualify_tables-12"><span class="linenos">12</span></a><span class="k">def</span> <span class="nf">qualify_tables</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-13"><a href="#qualify_tables-13"><span class="linenos">13</span></a> <span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-14"><a href="#qualify_tables-14"><span class="linenos">14</span></a> <span class="n">db</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-15"><a href="#qualify_tables-15"><span class="linenos">15</span></a> <span class="n">catalog</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-16"><a href="#qualify_tables-16"><span class="linenos">16</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">Schema</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
|
||||
</span><span id="qualify_tables-17"><a href="#qualify_tables-17"><span class="linenos">17</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-18"><a href="#qualify_tables-18"><span class="linenos">18</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="qualify_tables-19"><a href="#qualify_tables-19"><span class="linenos">19</span></a><span class="sd"> Rewrite sqlglot AST to have fully qualified tables. Join constructs such as</span>
|
||||
</span><span id="qualify_tables-20"><a href="#qualify_tables-20"><span class="linenos">20</span></a><span class="sd"> (t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</span>
|
||||
</span><span id="qualify_tables-21"><a href="#qualify_tables-21"><span class="linenos">21</span></a>
|
||||
</span><span id="qualify_tables-22"><a href="#qualify_tables-22"><span class="linenos">22</span></a><span class="sd"> Examples:</span>
|
||||
</span><span id="qualify_tables-23"><a href="#qualify_tables-23"><span class="linenos">23</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="qualify_tables-24"><a href="#qualify_tables-24"><span class="linenos">24</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT 1 FROM tbl")</span>
|
||||
</span><span id="qualify_tables-25"><a href="#qualify_tables-25"><span class="linenos">25</span></a><span class="sd"> >>> qualify_tables(expression, db="db").sql()</span>
|
||||
</span><span id="qualify_tables-26"><a href="#qualify_tables-26"><span class="linenos">26</span></a><span class="sd"> 'SELECT 1 FROM db.tbl AS tbl'</span>
|
||||
</span><span id="qualify_tables-27"><a href="#qualify_tables-27"><span class="linenos">27</span></a><span class="sd"> >>></span>
|
||||
</span><span id="qualify_tables-28"><a href="#qualify_tables-28"><span class="linenos">28</span></a><span class="sd"> >>> expression = sqlglot.parse_one("SELECT 1 FROM (t1 JOIN t2) AS t")</span>
|
||||
</span><span id="qualify_tables-29"><a href="#qualify_tables-29"><span class="linenos">29</span></a><span class="sd"> >>> qualify_tables(expression).sql()</span>
|
||||
</span><span id="qualify_tables-30"><a href="#qualify_tables-30"><span class="linenos">30</span></a><span class="sd"> 'SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t'</span>
|
||||
</span><span id="qualify_tables-31"><a href="#qualify_tables-31"><span class="linenos">31</span></a>
|
||||
</span><span id="qualify_tables-32"><a href="#qualify_tables-32"><span class="linenos">32</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="qualify_tables-33"><a href="#qualify_tables-33"><span class="linenos">33</span></a><span class="sd"> expression: Expression to qualify</span>
|
||||
</span><span id="qualify_tables-34"><a href="#qualify_tables-34"><span class="linenos">34</span></a><span class="sd"> db: Database name</span>
|
||||
</span><span id="qualify_tables-35"><a href="#qualify_tables-35"><span class="linenos">35</span></a><span class="sd"> catalog: Catalog name</span>
|
||||
</span><span id="qualify_tables-36"><a href="#qualify_tables-36"><span class="linenos">36</span></a><span class="sd"> schema: A schema to populate</span>
|
||||
</span><span id="qualify_tables-37"><a href="#qualify_tables-37"><span class="linenos">37</span></a>
|
||||
</span><span id="qualify_tables-38"><a href="#qualify_tables-38"><span class="linenos">38</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="qualify_tables-39"><a href="#qualify_tables-39"><span class="linenos">39</span></a><span class="sd"> The qualified expression.</span>
|
||||
</span><span id="qualify_tables-40"><a href="#qualify_tables-40"><span class="linenos">40</span></a><span class="sd"> """</span>
|
||||
</span><span id="qualify_tables-41"><a href="#qualify_tables-41"><span class="linenos">41</span></a> <span class="n">next_alias_name</span> <span class="o">=</span> <span class="n">name_sequence</span><span class="p">(</span><span class="s2">"_q_"</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-42"><a href="#qualify_tables-42"><span class="linenos">42</span></a>
|
||||
</span><span id="qualify_tables-43"><a href="#qualify_tables-43"><span class="linenos">43</span></a> <span class="k">for</span> <span class="n">scope</span> <span class="ow">in</span> <span class="n">traverse_scope</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-44"><a href="#qualify_tables-44"><span class="linenos">44</span></a> <span class="k">for</span> <span class="n">derived_table</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">ctes</span><span class="p">,</span> <span class="n">scope</span><span class="o">.</span><span class="n">derived_tables</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-45"><a href="#qualify_tables-45"><span class="linenos">45</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">derived_table</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subquery</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-46"><a href="#qualify_tables-46"><span class="linenos">46</span></a> <span class="n">unnested</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">unnest</span><span class="p">()</span>
|
||||
</span><span id="qualify_tables-47"><a href="#qualify_tables-47"><span class="linenos">47</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unnested</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-48"><a href="#qualify_tables-48"><span class="linenos">48</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">unnested</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-49"><a href="#qualify_tables-49"><span class="linenos">49</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s2">"*"</span><span class="p">)</span><span class="o">.</span><span class="n">from_</span><span class="p">(</span><span class="n">unnested</span><span class="o">.</span><span class="n">copy</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">))</span>
|
||||
</span><span id="qualify_tables-50"><a href="#qualify_tables-50"><span class="linenos">50</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">joins</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-51"><a href="#qualify_tables-51"><span class="linenos">51</span></a>
|
||||
</span><span id="qualify_tables-52"><a href="#qualify_tables-52"><span class="linenos">52</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-53"><a href="#qualify_tables-53"><span class="linenos">53</span></a> <span class="n">alias_</span> <span class="o">=</span> <span class="n">next_alias_name</span><span class="p">()</span>
|
||||
</span><span id="qualify_tables-54"><a href="#qualify_tables-54"><span class="linenos">54</span></a> <span class="n">derived_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">alias_</span><span class="p">)))</span>
|
||||
</span><span id="qualify_tables-55"><a href="#qualify_tables-55"><span class="linenos">55</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">rename_source</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="n">alias_</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-56"><a href="#qualify_tables-56"><span class="linenos">56</span></a>
|
||||
</span><span id="qualify_tables-57"><a href="#qualify_tables-57"><span class="linenos">57</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">derived_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-58"><a href="#qualify_tables-58"><span class="linenos">58</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-59"><a href="#qualify_tables-59"><span class="linenos">59</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())))</span>
|
||||
</span><span id="qualify_tables-60"><a href="#qualify_tables-60"><span class="linenos">60</span></a>
|
||||
</span><span id="qualify_tables-61"><a href="#qualify_tables-61"><span class="linenos">61</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">source</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="qualify_tables-62"><a href="#qualify_tables-62"><span class="linenos">62</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-63"><a href="#qualify_tables-63"><span class="linenos">63</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-64"><a href="#qualify_tables-64"><span class="linenos">64</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"db"</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-65"><a href="#qualify_tables-65"><span class="linenos">65</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"db"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">db</span><span class="p">))</span>
|
||||
</span><span id="qualify_tables-66"><a href="#qualify_tables-66"><span class="linenos">66</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-67"><a href="#qualify_tables-67"><span class="linenos">67</span></a> <span class="n">source</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"catalog"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">catalog</span><span class="p">))</span>
|
||||
</span><span id="qualify_tables-68"><a href="#qualify_tables-68"><span class="linenos">68</span></a>
|
||||
</span><span id="qualify_tables-69"><a href="#qualify_tables-69"><span class="linenos">69</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-70"><a href="#qualify_tables-70"><span class="linenos">70</span></a> <span class="c1"># Mutates the source by attaching an alias to it</span>
|
||||
</span><span id="qualify_tables-71"><a href="#qualify_tables-71"><span class="linenos">71</span></a> <span class="n">alias</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">name</span> <span class="ow">or</span> <span class="n">source</span><span class="o">.</span><span class="n">name</span> <span class="ow">or</span> <span class="n">next_alias_name</span><span class="p">(),</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-72"><a href="#qualify_tables-72"><span class="linenos">72</span></a>
|
||||
</span><span id="qualify_tables-73"><a href="#qualify_tables-73"><span class="linenos">73</span></a> <span class="n">pivots</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"pivots"</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-74"><a href="#qualify_tables-74"><span class="linenos">74</span></a> <span class="k">if</span> <span class="n">pivots</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-75"><a href="#qualify_tables-75"><span class="linenos">75</span></a> <span class="n">pivots</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-76"><a href="#qualify_tables-76"><span class="linenos">76</span></a> <span class="s2">"alias"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="qualify_tables-77"><a href="#qualify_tables-77"><span class="linenos">77</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-78"><a href="#qualify_tables-78"><span class="linenos">78</span></a>
|
||||
</span><span id="qualify_tables-79"><a href="#qualify_tables-79"><span class="linenos">79</span></a> <span class="k">if</span> <span class="n">schema</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">ReadCSV</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-80"><a href="#qualify_tables-80"><span class="linenos">80</span></a> <span class="k">with</span> <span class="n">csv_reader</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="k">as</span> <span class="n">reader</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-81"><a href="#qualify_tables-81"><span class="linenos">81</span></a> <span class="n">header</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-82"><a href="#qualify_tables-82"><span class="linenos">82</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">reader</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-83"><a href="#qualify_tables-83"><span class="linenos">83</span></a> <span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-84"><a href="#qualify_tables-84"><span class="linenos">84</span></a> <span class="n">source</span><span class="p">,</span> <span class="p">{</span><span class="n">k</span><span class="p">:</span> <span class="nb">type</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">.</span><span class="vm">__name__</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">header</span><span class="p">,</span> <span class="n">columns</span><span class="p">)}</span>
|
||||
</span><span id="qualify_tables-85"><a href="#qualify_tables-85"><span class="linenos">85</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-86"><a href="#qualify_tables-86"><span class="linenos">86</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="n">source</span><span class="o">.</span><span class="n">is_udtf</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-87"><a href="#qualify_tables-87"><span class="linenos">87</span></a> <span class="n">udtf</span> <span class="o">=</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="qualify_tables-88"><a href="#qualify_tables-88"><span class="linenos">88</span></a> <span class="n">table_alias</span> <span class="o">=</span> <span class="n">udtf</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">TableAlias</span><span class="p">(</span>
|
||||
</span><span id="qualify_tables-89"><a href="#qualify_tables-89"><span class="linenos">89</span></a> <span class="n">this</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">())</span>
|
||||
</span><span id="qualify_tables-90"><a href="#qualify_tables-90"><span class="linenos">90</span></a> <span class="p">)</span>
|
||||
</span><span id="qualify_tables-91"><a href="#qualify_tables-91"><span class="linenos">91</span></a> <span class="n">udtf</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">,</span> <span class="n">table_alias</span><span class="p">)</span>
|
||||
</span><span id="qualify_tables-92"><a href="#qualify_tables-92"><span class="linenos">92</span></a>
|
||||
</span><span id="qualify_tables-93"><a href="#qualify_tables-93"><span class="linenos">93</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">name</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-94"><a href="#qualify_tables-94"><span class="linenos">94</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"this"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">next_alias_name</span><span class="p">()))</span>
|
||||
</span><span id="qualify_tables-95"><a href="#qualify_tables-95"><span class="linenos">95</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">udtf</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">table_alias</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="qualify_tables-96"><a href="#qualify_tables-96"><span class="linenos">96</span></a> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">e</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">udtf</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">expressions</span><span class="p">):</span>
|
||||
</span><span id="qualify_tables-97"><a href="#qualify_tables-97"><span class="linenos">97</span></a> <span class="n">table_alias</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"columns"</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="sa">f</span><span class="s2">"_col_</span><span class="si">{</span><span class="n">i</span><span class="si">}</span><span class="s2">"</span><span class="p">))</span>
|
||||
</span><span id="qualify_tables-98"><a href="#qualify_tables-98"><span class="linenos">98</span></a>
|
||||
</span><span id="qualify_tables-99"><a href="#qualify_tables-99"><span class="linenos">99</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Rewrite sqlglot AST to have fully qualified tables. Additionally, this
|
||||
replaces "join constructs" (*) by equivalent SELECT * subqueries.</p>
|
||||
<div class="docstring"><p>Rewrite sqlglot AST to have fully qualified tables. Join constructs such as
|
||||
(t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.</p>
|
||||
|
||||
<h6 id="examples">Examples:</h6>
|
||||
|
||||
|
|
@ -286,9 +272,9 @@ replaces "join constructs" (*) by equivalent SELECT * subqueries.</p>
|
|||
<span class="gp">>>> </span><span class="n">qualify_tables</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">db</span><span class="o">=</span><span class="s2">"db"</span><span class="p">)</span><span class="o">.</span><span class="n">sql</span><span class="p">()</span>
|
||||
<span class="go">'SELECT 1 FROM db.tbl AS tbl'</span>
|
||||
<span class="gp">>>></span>
|
||||
<span class="gp">>>> </span><span class="n">expression</span> <span class="o">=</span> <span class="n"><a href="../../sqlglot.html#parse_one">sqlglot.parse_one</a></span><span class="p">(</span><span class="s2">"SELECT * FROM (tbl1 JOIN tbl2 ON id1 = id2)"</span><span class="p">)</span>
|
||||
<span class="gp">>>> </span><span class="n">expression</span> <span class="o">=</span> <span class="n"><a href="../../sqlglot.html#parse_one">sqlglot.parse_one</a></span><span class="p">(</span><span class="s2">"SELECT 1 FROM (t1 JOIN t2) AS t"</span><span class="p">)</span>
|
||||
<span class="gp">>>> </span><span class="n">qualify_tables</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span><span class="o">.</span><span class="n">sql</span><span class="p">()</span>
|
||||
<span class="go">'SELECT * FROM (SELECT * FROM tbl1 AS tbl1 JOIN tbl2 AS tbl2 ON id1 = id2) AS _q_0'</span>
|
||||
<span class="go">'SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t'</span>
|
||||
</code></pre>
|
||||
</div>
|
||||
</blockquote>
|
||||
|
|
@ -307,8 +293,6 @@ replaces "join constructs" (*) by equivalent SELECT * subqueries.</p>
|
|||
<blockquote>
|
||||
<p>The qualified expression.</p>
|
||||
</blockquote>
|
||||
|
||||
<p>(*) See section 7.2.1.2 in <a href="https://www.postgresql.org/docs/current/queries-table-expressions.html">https://www.postgresql.org/docs/current/queries-table-expressions.html</a></p>
|
||||
</div>
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
11916
docs/sqlglot/parser.html
11916
docs/sqlglot/parser.html
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -620,7 +620,16 @@ def concat_to_dpipe_sql(self: Generator, expression: exp.Concat | exp.SafeConcat
|
|||
return self.sql(this)
|
||||
|
||||
|
||||
# Spark, DuckDB use (almost) the same naming scheme for the output columns of the PIVOT operator
|
||||
def regexp_extract_sql(self: Generator, expression: exp.RegexpExtract) -> str:
|
||||
bad_args = list(filter(expression.args.get, ("position", "occurrence", "parameters")))
|
||||
if bad_args:
|
||||
self.unsupported(f"REGEXP_EXTRACT does not support the following arg(s): {bad_args}")
|
||||
|
||||
return self.func(
|
||||
"REGEXP_EXTRACT", expression.this, expression.expression, expression.args.get("group")
|
||||
)
|
||||
|
||||
|
||||
def pivot_column_names(aggregations: t.List[exp.Expression], dialect: DialectType) -> t.List[str]:
|
||||
names = []
|
||||
for agg in aggregations:
|
||||
|
|
|
|||
|
|
@ -15,6 +15,7 @@ from sqlglot.dialects.dialect import (
|
|||
no_properties_sql,
|
||||
no_safe_divide_sql,
|
||||
pivot_column_names,
|
||||
regexp_extract_sql,
|
||||
rename_func,
|
||||
str_position_sql,
|
||||
str_to_time_sql,
|
||||
|
|
@ -88,19 +89,6 @@ def _datatype_sql(self: generator.Generator, expression: exp.DataType) -> str:
|
|||
return self.datatype_sql(expression)
|
||||
|
||||
|
||||
def _regexp_extract_sql(self: generator.Generator, expression: exp.RegexpExtract) -> str:
|
||||
bad_args = list(filter(expression.args.get, ("position", "occurrence")))
|
||||
if bad_args:
|
||||
self.unsupported(f"REGEXP_EXTRACT does not support arg(s) {bad_args}")
|
||||
|
||||
return self.func(
|
||||
"REGEXP_EXTRACT",
|
||||
expression.args.get("this"),
|
||||
expression.args.get("expression"),
|
||||
expression.args.get("group"),
|
||||
)
|
||||
|
||||
|
||||
def _json_format_sql(self: generator.Generator, expression: exp.JSONFormat) -> str:
|
||||
sql = self.func("TO_JSON", expression.this, expression.args.get("options"))
|
||||
return f"CAST({sql} AS TEXT)"
|
||||
|
|
@ -156,6 +144,9 @@ class DuckDB(Dialect):
|
|||
"LIST_REVERSE_SORT": _sort_array_reverse,
|
||||
"LIST_SORT": exp.SortArray.from_arg_list,
|
||||
"LIST_VALUE": exp.Array.from_arg_list,
|
||||
"REGEXP_EXTRACT": lambda args: exp.RegexpExtract(
|
||||
this=seq_get(args, 0), expression=seq_get(args, 1), group=seq_get(args, 2)
|
||||
),
|
||||
"REGEXP_MATCHES": exp.RegexpLike.from_arg_list,
|
||||
"STRFTIME": format_time_lambda(exp.TimeToStr, "duckdb"),
|
||||
"STRING_SPLIT": exp.Split.from_arg_list,
|
||||
|
|
@ -227,7 +218,7 @@ class DuckDB(Dialect):
|
|||
exp.LogicalOr: rename_func("BOOL_OR"),
|
||||
exp.LogicalAnd: rename_func("BOOL_AND"),
|
||||
exp.Properties: no_properties_sql,
|
||||
exp.RegexpExtract: _regexp_extract_sql,
|
||||
exp.RegexpExtract: regexp_extract_sql,
|
||||
exp.RegexpLike: rename_func("REGEXP_MATCHES"),
|
||||
exp.RegexpSplit: rename_func("STR_SPLIT_REGEX"),
|
||||
exp.SafeDivide: no_safe_divide_sql,
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@ from sqlglot.dialects.dialect import (
|
|||
no_recursive_cte_sql,
|
||||
no_safe_divide_sql,
|
||||
no_trycast_sql,
|
||||
regexp_extract_sql,
|
||||
rename_func,
|
||||
right_to_substring_sql,
|
||||
strposition_to_locate_sql,
|
||||
|
|
@ -230,24 +231,25 @@ class Hive(Dialect):
|
|||
**parser.Parser.FUNCTIONS,
|
||||
"BASE64": exp.ToBase64.from_arg_list,
|
||||
"COLLECT_LIST": exp.ArrayAgg.from_arg_list,
|
||||
"COLLECT_SET": exp.SetAgg.from_arg_list,
|
||||
"DATE_ADD": lambda args: exp.TsOrDsAdd(
|
||||
this=seq_get(args, 0), expression=seq_get(args, 1), unit=exp.Literal.string("DAY")
|
||||
),
|
||||
"DATEDIFF": lambda args: exp.DateDiff(
|
||||
this=exp.TsOrDsToDate(this=seq_get(args, 0)),
|
||||
expression=exp.TsOrDsToDate(this=seq_get(args, 1)),
|
||||
),
|
||||
"DATE_SUB": lambda args: exp.TsOrDsAdd(
|
||||
this=seq_get(args, 0),
|
||||
expression=exp.Mul(this=seq_get(args, 1), expression=exp.Literal.number(-1)),
|
||||
unit=exp.Literal.string("DAY"),
|
||||
),
|
||||
"DATE_FORMAT": lambda args: format_time_lambda(exp.TimeToStr, "hive")(
|
||||
[
|
||||
exp.TimeStrToTime(this=seq_get(args, 0)),
|
||||
seq_get(args, 1),
|
||||
]
|
||||
),
|
||||
"DATE_SUB": lambda args: exp.TsOrDsAdd(
|
||||
this=seq_get(args, 0),
|
||||
expression=exp.Mul(this=seq_get(args, 1), expression=exp.Literal.number(-1)),
|
||||
unit=exp.Literal.string("DAY"),
|
||||
),
|
||||
"DATEDIFF": lambda args: exp.DateDiff(
|
||||
this=exp.TsOrDsToDate(this=seq_get(args, 0)),
|
||||
expression=exp.TsOrDsToDate(this=seq_get(args, 1)),
|
||||
),
|
||||
"DAY": lambda args: exp.Day(this=exp.TsOrDsToDate(this=seq_get(args, 0))),
|
||||
"FROM_UNIXTIME": format_time_lambda(exp.UnixToStr, "hive", True),
|
||||
"GET_JSON_OBJECT": exp.JSONExtractScalar.from_arg_list,
|
||||
|
|
@ -256,7 +258,9 @@ class Hive(Dialect):
|
|||
"MONTH": lambda args: exp.Month(this=exp.TsOrDsToDate.from_arg_list(args)),
|
||||
"PERCENTILE": exp.Quantile.from_arg_list,
|
||||
"PERCENTILE_APPROX": exp.ApproxQuantile.from_arg_list,
|
||||
"COLLECT_SET": exp.SetAgg.from_arg_list,
|
||||
"REGEXP_EXTRACT": lambda args: exp.RegexpExtract(
|
||||
this=seq_get(args, 0), expression=seq_get(args, 1), group=seq_get(args, 2)
|
||||
),
|
||||
"SIZE": exp.ArraySize.from_arg_list,
|
||||
"SPLIT": exp.RegexpSplit.from_arg_list,
|
||||
"TO_DATE": format_time_lambda(exp.TsOrDsToDate, "hive"),
|
||||
|
|
@ -363,6 +367,7 @@ class Hive(Dialect):
|
|||
exp.Create: create_with_partitions_sql,
|
||||
exp.Quantile: rename_func("PERCENTILE"),
|
||||
exp.ApproxQuantile: rename_func("PERCENTILE_APPROX"),
|
||||
exp.RegexpExtract: regexp_extract_sql,
|
||||
exp.RegexpLike: lambda self, e: self.binary(e, "RLIKE"),
|
||||
exp.RegexpSplit: rename_func("SPLIT"),
|
||||
exp.Right: right_to_substring_sql,
|
||||
|
|
@ -422,5 +427,12 @@ class Hive(Dialect):
|
|||
expression = exp.DataType.build("text")
|
||||
elif expression.this in exp.DataType.TEMPORAL_TYPES:
|
||||
expression = exp.DataType.build(expression.this)
|
||||
elif expression.is_type("float"):
|
||||
size_expression = expression.find(exp.DataTypeSize)
|
||||
if size_expression:
|
||||
size = int(size_expression.name)
|
||||
expression = (
|
||||
exp.DataType.build("float") if size <= 32 else exp.DataType.build("double")
|
||||
)
|
||||
|
||||
return super().datatype_sql(expression)
|
||||
|
|
|
|||
|
|
@ -193,6 +193,12 @@ class MySQL(Dialect):
|
|||
TokenType.VALUES,
|
||||
}
|
||||
|
||||
CONJUNCTION = {
|
||||
**parser.Parser.CONJUNCTION,
|
||||
TokenType.DAMP: exp.And,
|
||||
TokenType.XOR: exp.Xor,
|
||||
}
|
||||
|
||||
TABLE_ALIAS_TOKENS = (
|
||||
parser.Parser.TABLE_ALIAS_TOKENS - parser.Parser.TABLE_INDEX_HINT_TOKENS
|
||||
)
|
||||
|
|
|
|||
|
|
@ -99,6 +99,9 @@ class Oracle(Dialect):
|
|||
LOCKING_READS_SUPPORTED = True
|
||||
JOIN_HINTS = False
|
||||
TABLE_HINTS = False
|
||||
COLUMN_JOIN_MARKS_SUPPORTED = True
|
||||
|
||||
LIMIT_FETCH = "FETCH"
|
||||
|
||||
TYPE_MAPPING = {
|
||||
**generator.Generator.TYPE_MAPPING,
|
||||
|
|
@ -110,6 +113,7 @@ class Oracle(Dialect):
|
|||
exp.DataType.Type.DOUBLE: "DOUBLE PRECISION",
|
||||
exp.DataType.Type.VARCHAR: "VARCHAR2",
|
||||
exp.DataType.Type.NVARCHAR: "NVARCHAR2",
|
||||
exp.DataType.Type.NCHAR: "NCHAR",
|
||||
exp.DataType.Type.TEXT: "CLOB",
|
||||
exp.DataType.Type.BINARY: "BLOB",
|
||||
exp.DataType.Type.VARBINARY: "BLOB",
|
||||
|
|
@ -140,15 +144,9 @@ class Oracle(Dialect):
|
|||
exp.VolatileProperty: exp.Properties.Location.UNSUPPORTED,
|
||||
}
|
||||
|
||||
LIMIT_FETCH = "FETCH"
|
||||
|
||||
def offset_sql(self, expression: exp.Offset) -> str:
|
||||
return f"{super().offset_sql(expression)} ROWS"
|
||||
|
||||
def column_sql(self, expression: exp.Column) -> str:
|
||||
column = super().column_sql(expression)
|
||||
return f"{column} (+)" if expression.args.get("join_mark") else column
|
||||
|
||||
def xmltable_sql(self, expression: exp.XMLTable) -> str:
|
||||
this = self.sql(expression, "this")
|
||||
passing = self.expressions(expression, key="passing")
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ from sqlglot.dialects.dialect import (
|
|||
no_ilike_sql,
|
||||
no_pivot_sql,
|
||||
no_safe_divide_sql,
|
||||
regexp_extract_sql,
|
||||
rename_func,
|
||||
right_to_substring_sql,
|
||||
struct_extract_sql,
|
||||
|
|
@ -215,6 +216,9 @@ class Presto(Dialect):
|
|||
this=seq_get(args, 0), replace=seq_get(args, 1), charset=exp.Literal.string("utf-8")
|
||||
),
|
||||
"NOW": exp.CurrentTimestamp.from_arg_list,
|
||||
"REGEXP_EXTRACT": lambda args: exp.RegexpExtract(
|
||||
this=seq_get(args, 0), expression=seq_get(args, 1), group=seq_get(args, 2)
|
||||
),
|
||||
"SEQUENCE": exp.GenerateSeries.from_arg_list,
|
||||
"STRPOS": lambda args: exp.StrPosition(
|
||||
this=seq_get(args, 0), substr=seq_get(args, 1), instance=seq_get(args, 2)
|
||||
|
|
@ -293,6 +297,7 @@ class Presto(Dialect):
|
|||
exp.LogicalOr: rename_func("BOOL_OR"),
|
||||
exp.Pivot: no_pivot_sql,
|
||||
exp.Quantile: _quantile_sql,
|
||||
exp.RegexpExtract: regexp_extract_sql,
|
||||
exp.Right: right_to_substring_sql,
|
||||
exp.SafeBracket: lambda self, e: self.func(
|
||||
"ELEMENT_AT", e.this, seq_get(apply_index_offset(e.this, e.expressions, 1), 0)
|
||||
|
|
|
|||
|
|
@ -223,13 +223,14 @@ class Snowflake(Dialect):
|
|||
"IFF": exp.If.from_arg_list,
|
||||
"NULLIFZERO": _nullifzero_to_if,
|
||||
"OBJECT_CONSTRUCT": _parse_object_construct,
|
||||
"REGEXP_SUBSTR": exp.RegexpExtract.from_arg_list,
|
||||
"RLIKE": exp.RegexpLike.from_arg_list,
|
||||
"SQUARE": lambda args: exp.Pow(this=seq_get(args, 0), expression=exp.Literal.number(2)),
|
||||
"TIMEDIFF": _parse_datediff,
|
||||
"TIMESTAMPDIFF": _parse_datediff,
|
||||
"TO_ARRAY": exp.Array.from_arg_list,
|
||||
"TO_VARCHAR": exp.ToChar.from_arg_list,
|
||||
"TO_TIMESTAMP": _snowflake_to_timestamp,
|
||||
"TO_VARCHAR": exp.ToChar.from_arg_list,
|
||||
"ZEROIFNULL": _zeroifnull_to_if,
|
||||
}
|
||||
|
||||
|
|
@ -361,12 +362,12 @@ class Snowflake(Dialect):
|
|||
"OBJECT_CONSTRUCT",
|
||||
*(arg for expression in e.expressions for arg in expression.flatten()),
|
||||
),
|
||||
exp.TimestampTrunc: timestamptrunc_sql,
|
||||
exp.TimeStrToTime: timestrtotime_sql,
|
||||
exp.TimeToUnix: lambda self, e: f"EXTRACT(epoch_second FROM {self.sql(e, 'this')})",
|
||||
exp.TimeToStr: lambda self, e: self.func(
|
||||
"TO_CHAR", exp.cast(e.this, "timestamp"), self.format_time(e)
|
||||
),
|
||||
exp.TimestampTrunc: timestamptrunc_sql,
|
||||
exp.TimeToUnix: lambda self, e: f"EXTRACT(epoch_second FROM {self.sql(e, 'this')})",
|
||||
exp.ToChar: lambda self, e: self.function_fallback_sql(e),
|
||||
exp.Trim: lambda self, e: self.func("TRIM", e.this, e.expression),
|
||||
exp.TsOrDsToDate: ts_or_ds_to_date_sql("snowflake"),
|
||||
|
|
@ -390,6 +391,24 @@ class Snowflake(Dialect):
|
|||
exp.VolatileProperty: exp.Properties.Location.UNSUPPORTED,
|
||||
}
|
||||
|
||||
def regexpextract_sql(self, expression: exp.RegexpExtract) -> str:
|
||||
# Other dialects don't support all of the following parameters, so we need to
|
||||
# generate default values as necessary to ensure the transpilation is correct
|
||||
group = expression.args.get("group")
|
||||
parameters = expression.args.get("parameters") or (group and exp.Literal.string("c"))
|
||||
occurrence = expression.args.get("occurrence") or (parameters and exp.Literal.number(1))
|
||||
position = expression.args.get("position") or (occurrence and exp.Literal.number(1))
|
||||
|
||||
return self.func(
|
||||
"REGEXP_SUBSTR",
|
||||
expression.this,
|
||||
expression.expression,
|
||||
position,
|
||||
occurrence,
|
||||
parameters,
|
||||
group,
|
||||
)
|
||||
|
||||
def except_op(self, expression: exp.Except) -> str:
|
||||
if not expression.args.get("distinct", False):
|
||||
self.unsupported("EXCEPT with All is not supported in Snowflake")
|
||||
|
|
|
|||
|
|
@ -302,6 +302,7 @@ class TSQL(Dialect):
|
|||
"UNIQUEIDENTIFIER": TokenType.UNIQUEIDENTIFIER,
|
||||
"VARCHAR(MAX)": TokenType.TEXT,
|
||||
"XML": TokenType.XML,
|
||||
"OUTPUT": TokenType.RETURNING,
|
||||
"SYSTEM_USER": TokenType.CURRENT_USER,
|
||||
}
|
||||
|
||||
|
|
@ -469,6 +470,7 @@ class TSQL(Dialect):
|
|||
LOCKING_READS_SUPPORTED = True
|
||||
LIMIT_IS_TOP = True
|
||||
QUERY_HINTS = False
|
||||
RETURNING_END = False
|
||||
|
||||
TYPE_MAPPING = {
|
||||
**generator.Generator.TYPE_MAPPING,
|
||||
|
|
@ -532,3 +534,8 @@ class TSQL(Dialect):
|
|||
table = expression.args.get("table")
|
||||
table = f"{table} " if table else ""
|
||||
return f"RETURNS {table}{self.sql(expression, 'this')}"
|
||||
|
||||
def returning_sql(self, expression: exp.Returning) -> str:
|
||||
into = self.sql(expression, "into")
|
||||
into = self.seg(f"INTO {into}") if into else ""
|
||||
return f"{self.seg('OUTPUT')} {self.expressions(expression, flat=True)}{into}"
|
||||
|
|
|
|||
|
|
@ -878,11 +878,11 @@ class DerivedTable(Expression):
|
|||
return [c.name for c in table_alias.args.get("columns") or []]
|
||||
|
||||
@property
|
||||
def selects(self):
|
||||
def selects(self) -> t.List[Expression]:
|
||||
return self.this.selects if isinstance(self.this, Subqueryable) else []
|
||||
|
||||
@property
|
||||
def named_selects(self):
|
||||
def named_selects(self) -> t.List[str]:
|
||||
return [select.output_name for select in self.selects]
|
||||
|
||||
|
||||
|
|
@ -959,7 +959,7 @@ class Unionable(Expression):
|
|||
|
||||
class UDTF(DerivedTable, Unionable):
|
||||
@property
|
||||
def selects(self):
|
||||
def selects(self) -> t.List[Expression]:
|
||||
alias = self.args.get("alias")
|
||||
return alias.columns if alias else []
|
||||
|
||||
|
|
@ -1576,7 +1576,7 @@ class OnConflict(Expression):
|
|||
|
||||
|
||||
class Returning(Expression):
|
||||
arg_types = {"expressions": True}
|
||||
arg_types = {"expressions": True, "into": False}
|
||||
|
||||
|
||||
# https://dev.mysql.com/doc/refman/8.0/en/charset-introducer.html
|
||||
|
|
@ -2194,11 +2194,11 @@ class Subqueryable(Unionable):
|
|||
return with_.expressions
|
||||
|
||||
@property
|
||||
def selects(self):
|
||||
def selects(self) -> t.List[Expression]:
|
||||
raise NotImplementedError("Subqueryable objects must implement `selects`")
|
||||
|
||||
@property
|
||||
def named_selects(self):
|
||||
def named_selects(self) -> t.List[str]:
|
||||
raise NotImplementedError("Subqueryable objects must implement `named_selects`")
|
||||
|
||||
def with_(
|
||||
|
|
@ -2282,7 +2282,6 @@ class Table(Expression):
|
|||
"pivots": False,
|
||||
"hints": False,
|
||||
"system_time": False,
|
||||
"wrapped": False,
|
||||
}
|
||||
|
||||
@property
|
||||
|
|
@ -2299,14 +2298,28 @@ class Table(Expression):
|
|||
def catalog(self) -> str:
|
||||
return self.text("catalog")
|
||||
|
||||
@property
|
||||
def selects(self) -> t.List[Expression]:
|
||||
return []
|
||||
|
||||
@property
|
||||
def named_selects(self) -> t.List[str]:
|
||||
return []
|
||||
|
||||
@property
|
||||
def parts(self) -> t.List[Identifier]:
|
||||
"""Return the parts of a table in order catalog, db, table."""
|
||||
return [
|
||||
t.cast(Identifier, self.args[part])
|
||||
for part in ("catalog", "db", "this")
|
||||
if self.args.get(part)
|
||||
]
|
||||
parts: t.List[Identifier] = []
|
||||
|
||||
for arg in ("catalog", "db", "this"):
|
||||
part = self.args.get(arg)
|
||||
|
||||
if isinstance(part, Identifier):
|
||||
parts.append(part)
|
||||
elif isinstance(part, Dot):
|
||||
parts.extend(part.flatten())
|
||||
|
||||
return parts
|
||||
|
||||
|
||||
# See the TSQL "Querying data in a system-versioned temporal table" page
|
||||
|
|
@ -2390,7 +2403,7 @@ class Union(Subqueryable):
|
|||
return this
|
||||
|
||||
@property
|
||||
def named_selects(self):
|
||||
def named_selects(self) -> t.List[str]:
|
||||
return self.this.unnest().named_selects
|
||||
|
||||
@property
|
||||
|
|
@ -2398,7 +2411,7 @@ class Union(Subqueryable):
|
|||
return self.this.is_star or self.expression.is_star
|
||||
|
||||
@property
|
||||
def selects(self):
|
||||
def selects(self) -> t.List[Expression]:
|
||||
return self.this.unnest().selects
|
||||
|
||||
@property
|
||||
|
|
@ -3517,6 +3530,10 @@ class Or(Connector):
|
|||
pass
|
||||
|
||||
|
||||
class Xor(Connector):
|
||||
pass
|
||||
|
||||
|
||||
class BitwiseAnd(Binary):
|
||||
pass
|
||||
|
||||
|
|
@ -4409,6 +4426,7 @@ class RegexpExtract(Func):
|
|||
"expression": True,
|
||||
"position": False,
|
||||
"occurrence": False,
|
||||
"parameters": False,
|
||||
"group": False,
|
||||
}
|
||||
|
||||
|
|
@ -5756,7 +5774,9 @@ def table_name(table: Table | str, dialect: DialectType = None) -> str:
|
|||
raise ValueError(f"Cannot parse {table}")
|
||||
|
||||
return ".".join(
|
||||
part.sql(dialect=dialect) if not SAFE_IDENTIFIER_RE.match(part.name) else part.name
|
||||
part.sql(dialect=dialect, identify=True)
|
||||
if not SAFE_IDENTIFIER_RE.match(part.name)
|
||||
else part.name
|
||||
for part in table.parts
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -155,6 +155,12 @@ class Generator:
|
|||
# Whether or not to generate the limit as TOP <value> instead of LIMIT <value>
|
||||
LIMIT_IS_TOP = False
|
||||
|
||||
# Whether or not to generate INSERT INTO ... RETURNING or INSERT INTO RETURNING ...
|
||||
RETURNING_END = True
|
||||
|
||||
# Whether or not to generate the (+) suffix for columns used in old-style join conditions
|
||||
COLUMN_JOIN_MARKS_SUPPORTED = False
|
||||
|
||||
# https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax
|
||||
SELECT_KINDS: t.Tuple[str, ...] = ("STRUCT", "VALUE")
|
||||
|
||||
|
|
@ -556,7 +562,13 @@ class Generator:
|
|||
return f"{default}CHARACTER SET={self.sql(expression, 'this')}"
|
||||
|
||||
def column_sql(self, expression: exp.Column) -> str:
|
||||
return ".".join(
|
||||
join_mark = " (+)" if expression.args.get("join_mark") else ""
|
||||
|
||||
if join_mark and not self.COLUMN_JOIN_MARKS_SUPPORTED:
|
||||
join_mark = ""
|
||||
self.unsupported("Outer join syntax using the (+) operator is not supported.")
|
||||
|
||||
column = ".".join(
|
||||
self.sql(part)
|
||||
for part in (
|
||||
expression.args.get("catalog"),
|
||||
|
|
@ -567,6 +579,8 @@ class Generator:
|
|||
if part
|
||||
)
|
||||
|
||||
return f"{column}{join_mark}"
|
||||
|
||||
def columnposition_sql(self, expression: exp.ColumnPosition) -> str:
|
||||
this = self.sql(expression, "this")
|
||||
this = f" {this}" if this else ""
|
||||
|
|
@ -836,8 +850,11 @@ class Generator:
|
|||
limit = self.sql(expression, "limit")
|
||||
tables = self.expressions(expression, key="tables")
|
||||
tables = f" {tables}" if tables else ""
|
||||
sql = f"DELETE{tables}{this}{using}{where}{returning}{limit}"
|
||||
return self.prepend_ctes(expression, sql)
|
||||
if self.RETURNING_END:
|
||||
expression_sql = f"{this}{using}{where}{returning}{limit}"
|
||||
else:
|
||||
expression_sql = f"{returning}{this}{using}{where}{limit}"
|
||||
return self.prepend_ctes(expression, f"DELETE{tables}{expression_sql}")
|
||||
|
||||
def drop_sql(self, expression: exp.Drop) -> str:
|
||||
this = self.sql(expression, "this")
|
||||
|
|
@ -887,7 +904,8 @@ class Generator:
|
|||
unique = "UNIQUE " if expression.args.get("unique") else ""
|
||||
primary = "PRIMARY " if expression.args.get("primary") else ""
|
||||
amp = "AMP " if expression.args.get("amp") else ""
|
||||
name = f"{expression.name} " if expression.name else ""
|
||||
name = self.sql(expression, "this")
|
||||
name = f"{name} " if name else ""
|
||||
table = self.sql(expression, "table")
|
||||
table = f"{self.INDEX_ON} {table} " if table else ""
|
||||
using = self.sql(expression, "using")
|
||||
|
|
@ -1134,7 +1152,13 @@ class Generator:
|
|||
expression_sql = f"{self.sep()}{self.sql(expression, 'expression')}"
|
||||
conflict = self.sql(expression, "conflict")
|
||||
returning = self.sql(expression, "returning")
|
||||
sql = f"INSERT{alternative}{ignore}{this}{exists}{partition_sql}{where}{expression_sql}{conflict}{returning}"
|
||||
|
||||
if self.RETURNING_END:
|
||||
expression_sql = f"{expression_sql}{conflict}{returning}"
|
||||
else:
|
||||
expression_sql = f"{returning}{expression_sql}{conflict}"
|
||||
|
||||
sql = f"INSERT{alternative}{ignore}{this}{exists}{partition_sql}{where}{expression_sql}"
|
||||
return self.prepend_ctes(expression, sql)
|
||||
|
||||
def intersect_sql(self, expression: exp.Intersect) -> str:
|
||||
|
|
@ -1215,8 +1239,7 @@ class Generator:
|
|||
system_time = expression.args.get("system_time")
|
||||
system_time = f" {self.sql(expression, 'system_time')}" if system_time else ""
|
||||
|
||||
sql = f"{table}{system_time}{alias}{hints}{pivots}{joins}{laterals}"
|
||||
return f"({sql})" if expression.args.get("wrapped") else sql
|
||||
return f"{table}{system_time}{alias}{hints}{pivots}{joins}{laterals}"
|
||||
|
||||
def tablesample_sql(
|
||||
self, expression: exp.TableSample, seed_prefix: str = "SEED", sep=" AS "
|
||||
|
|
@ -1276,7 +1299,11 @@ class Generator:
|
|||
where_sql = self.sql(expression, "where")
|
||||
returning = self.sql(expression, "returning")
|
||||
limit = self.sql(expression, "limit")
|
||||
sql = f"UPDATE {this} SET {set_sql}{from_sql}{where_sql}{returning}{limit}"
|
||||
if self.RETURNING_END:
|
||||
expression_sql = f"{from_sql}{where_sql}{returning}{limit}"
|
||||
else:
|
||||
expression_sql = f"{returning}{from_sql}{where_sql}{limit}"
|
||||
sql = f"UPDATE {this} SET {set_sql}{expression_sql}"
|
||||
return self.prepend_ctes(expression, sql)
|
||||
|
||||
def values_sql(self, expression: exp.Values) -> str:
|
||||
|
|
@ -2016,6 +2043,9 @@ class Generator:
|
|||
def and_sql(self, expression: exp.And) -> str:
|
||||
return self.connector_sql(expression, "AND")
|
||||
|
||||
def xor_sql(self, expression: exp.And) -> str:
|
||||
return self.connector_sql(expression, "XOR")
|
||||
|
||||
def connector_sql(self, expression: exp.Connector, op: str) -> str:
|
||||
if not self.pretty:
|
||||
return self.binary(expression, op)
|
||||
|
|
|
|||
|
|
@ -104,7 +104,7 @@ def lineage(
|
|||
# Find the specific select clause that is the source of the column we want.
|
||||
# This can either be a specific, named select or a generic `*` clause.
|
||||
select = next(
|
||||
(select for select in scope.selects if select.alias_or_name == column_name),
|
||||
(select for select in scope.expression.selects if select.alias_or_name == column_name),
|
||||
exp.Star() if scope.expression.is_star else None,
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -85,7 +85,7 @@ def _unique_outputs(scope):
|
|||
grouped_outputs = set()
|
||||
|
||||
unique_outputs = set()
|
||||
for select in scope.selects:
|
||||
for select in scope.expression.selects:
|
||||
output = select.unalias()
|
||||
if output in grouped_expressions:
|
||||
grouped_outputs.add(output)
|
||||
|
|
@ -105,7 +105,7 @@ def _unique_outputs(scope):
|
|||
|
||||
def _has_single_output_row(scope):
|
||||
return isinstance(scope.expression, exp.Select) and (
|
||||
all(isinstance(e.unalias(), exp.AggFunc) for e in scope.selects)
|
||||
all(isinstance(e.unalias(), exp.AggFunc) for e in scope.expression.selects)
|
||||
or _is_limit_1(scope)
|
||||
or not scope.expression.args.get("from")
|
||||
)
|
||||
|
|
|
|||
|
|
@ -113,7 +113,7 @@ def _eliminate_union(scope, existing_ctes, taken):
|
|||
taken[alias] = scope
|
||||
|
||||
# Try to maintain the selections
|
||||
expressions = scope.selects
|
||||
expressions = scope.expression.selects
|
||||
selects = [
|
||||
exp.alias_(exp.column(e.alias_or_name, table=alias), alias=e.alias_or_name, copy=False)
|
||||
for e in expressions
|
||||
|
|
|
|||
|
|
@ -12,7 +12,12 @@ def isolate_table_selects(expression, schema=None):
|
|||
continue
|
||||
|
||||
for _, source in scope.selected_sources.values():
|
||||
if not isinstance(source, exp.Table) or not schema.column_names(source):
|
||||
if (
|
||||
not isinstance(source, exp.Table)
|
||||
or not schema.column_names(source)
|
||||
or isinstance(source.parent, exp.Subquery)
|
||||
or isinstance(source.parent.parent, exp.Table)
|
||||
):
|
||||
continue
|
||||
|
||||
if not source.alias:
|
||||
|
|
|
|||
|
|
@ -107,6 +107,7 @@ def merge_derived_tables(expression, leave_tables_isolated=False):
|
|||
_merge_order(outer_scope, inner_scope)
|
||||
_merge_hints(outer_scope, inner_scope)
|
||||
outer_scope.clear_cache()
|
||||
|
||||
return expression
|
||||
|
||||
|
||||
|
|
@ -166,7 +167,7 @@ def _mergeable(outer_scope, inner_scope, leave_tables_isolated, from_or_join):
|
|||
if not inner_from:
|
||||
return False
|
||||
inner_from_table = inner_from.alias_or_name
|
||||
inner_projections = {s.alias_or_name: s for s in inner_scope.selects}
|
||||
inner_projections = {s.alias_or_name: s for s in inner_scope.expression.selects}
|
||||
return any(
|
||||
col.table != inner_from_table
|
||||
for selection in selections
|
||||
|
|
|
|||
|
|
@ -59,7 +59,7 @@ def reorder_joins(expression):
|
|||
dag = {name: other_table_names(join) for name, join in joins.items()}
|
||||
parent.set(
|
||||
"joins",
|
||||
[joins[name] for name in tsort(dag) if name != from_.alias_or_name],
|
||||
[joins[name] for name in tsort(dag) if name != from_.alias_or_name and name in joins],
|
||||
)
|
||||
return expression
|
||||
|
||||
|
|
|
|||
|
|
@ -42,7 +42,10 @@ def pushdown_predicates(expression):
|
|||
# so we limit the selected sources to only itself
|
||||
for join in select.args.get("joins") or []:
|
||||
name = join.alias_or_name
|
||||
pushdown(join.args.get("on"), {name: scope.selected_sources[name]}, scope_ref_count)
|
||||
if name in scope.selected_sources:
|
||||
pushdown(
|
||||
join.args.get("on"), {name: scope.selected_sources[name]}, scope_ref_count
|
||||
)
|
||||
|
||||
return expression
|
||||
|
||||
|
|
|
|||
|
|
@ -48,12 +48,12 @@ def pushdown_projections(expression, schema=None, remove_unused_selections=True)
|
|||
left, right = scope.union_scopes
|
||||
referenced_columns[left] = parent_selections
|
||||
|
||||
if any(select.is_star for select in right.selects):
|
||||
if any(select.is_star for select in right.expression.selects):
|
||||
referenced_columns[right] = parent_selections
|
||||
elif not any(select.is_star for select in left.selects):
|
||||
elif not any(select.is_star for select in left.expression.selects):
|
||||
referenced_columns[right] = [
|
||||
right.selects[i].alias_or_name
|
||||
for i, select in enumerate(left.selects)
|
||||
right.expression.selects[i].alias_or_name
|
||||
for i, select in enumerate(left.expression.selects)
|
||||
if SELECT_ALL in parent_selections or select.alias_or_name in parent_selections
|
||||
]
|
||||
|
||||
|
|
@ -90,7 +90,7 @@ def _remove_unused_selections(scope, parent_selections, schema):
|
|||
removed = False
|
||||
star = False
|
||||
|
||||
for selection in scope.selects:
|
||||
for selection in scope.expression.selects:
|
||||
name = selection.alias_or_name
|
||||
|
||||
if SELECT_ALL in parent_selections or name in parent_selections or name in order_refs:
|
||||
|
|
|
|||
|
|
@ -192,13 +192,13 @@ def _expand_alias_refs(scope: Scope, resolver: Resolver) -> None:
|
|||
if table and (not alias_expr or double_agg):
|
||||
column.set("table", table)
|
||||
elif not column.table and alias_expr and not double_agg:
|
||||
if isinstance(alias_expr, exp.Literal):
|
||||
if isinstance(alias_expr, exp.Literal) and (literal_index or resolve_table):
|
||||
if literal_index:
|
||||
column.replace(exp.Literal.number(i))
|
||||
else:
|
||||
column.replace(alias_expr.copy())
|
||||
|
||||
for i, projection in enumerate(scope.selects):
|
||||
for i, projection in enumerate(scope.expression.selects):
|
||||
replace_columns(projection)
|
||||
|
||||
if isinstance(projection, exp.Alias):
|
||||
|
|
@ -239,7 +239,7 @@ def _expand_order_by(scope: Scope, resolver: Resolver):
|
|||
ordered.set("this", new_expression)
|
||||
|
||||
if scope.expression.args.get("group"):
|
||||
selects = {s.this: exp.column(s.alias_or_name) for s in scope.selects}
|
||||
selects = {s.this: exp.column(s.alias_or_name) for s in scope.expression.selects}
|
||||
|
||||
for ordered in ordereds:
|
||||
ordered = ordered.this
|
||||
|
|
@ -270,7 +270,7 @@ def _expand_positional_references(scope: Scope, expressions: t.Iterable[E]) -> t
|
|||
|
||||
def _select_by_pos(scope: Scope, node: exp.Literal) -> exp.Alias:
|
||||
try:
|
||||
return scope.selects[int(node.this) - 1].assert_is(exp.Alias)
|
||||
return scope.expression.selects[int(node.this) - 1].assert_is(exp.Alias)
|
||||
except IndexError:
|
||||
raise OptimizeError(f"Unknown output column: {node.name}")
|
||||
|
||||
|
|
@ -347,7 +347,7 @@ def _expand_stars(
|
|||
if not pivot_output_columns:
|
||||
pivot_output_columns = [col.alias_or_name for col in pivot.expressions]
|
||||
|
||||
for expression in scope.selects:
|
||||
for expression in scope.expression.selects:
|
||||
if isinstance(expression, exp.Star):
|
||||
tables = list(scope.selected_sources)
|
||||
_add_except_columns(expression, tables, except_columns)
|
||||
|
|
@ -446,7 +446,7 @@ def _qualify_outputs(scope: Scope):
|
|||
new_selections = []
|
||||
|
||||
for i, (selection, aliased_column) in enumerate(
|
||||
itertools.zip_longest(scope.selects, scope.outer_column_list)
|
||||
itertools.zip_longest(scope.expression.selects, scope.outer_column_list)
|
||||
):
|
||||
if isinstance(selection, exp.Subquery):
|
||||
if not selection.output_name:
|
||||
|
|
|
|||
|
|
@ -15,7 +15,8 @@ def qualify_tables(
|
|||
schema: t.Optional[Schema] = None,
|
||||
) -> E:
|
||||
"""
|
||||
Rewrite sqlglot AST to have fully qualified, unnested tables.
|
||||
Rewrite sqlglot AST to have fully qualified tables. Join constructs such as
|
||||
(t1 JOIN t2) AS t will be expanded into (SELECT * FROM t1 AS t1, t2 AS t2) AS t.
|
||||
|
||||
Examples:
|
||||
>>> import sqlglot
|
||||
|
|
@ -23,18 +24,9 @@ def qualify_tables(
|
|||
>>> qualify_tables(expression, db="db").sql()
|
||||
'SELECT 1 FROM db.tbl AS tbl'
|
||||
>>>
|
||||
>>> expression = sqlglot.parse_one("SELECT * FROM (tbl)")
|
||||
>>> expression = sqlglot.parse_one("SELECT 1 FROM (t1 JOIN t2) AS t")
|
||||
>>> qualify_tables(expression).sql()
|
||||
'SELECT * FROM tbl AS tbl'
|
||||
>>>
|
||||
>>> expression = sqlglot.parse_one("SELECT * FROM (tbl1 JOIN tbl2 ON id1 = id2)")
|
||||
>>> qualify_tables(expression).sql()
|
||||
'SELECT * FROM tbl1 AS tbl1 JOIN tbl2 AS tbl2 ON id1 = id2'
|
||||
|
||||
Note:
|
||||
This rule effectively enforces a left-to-right join order, since all joins
|
||||
are unnested. This means that the optimizer doesn't necessarily preserve the
|
||||
original join order, e.g. when parentheses are used to specify it explicitly.
|
||||
'SELECT 1 FROM (SELECT * FROM t1 AS t1, t2 AS t2) AS t'
|
||||
|
||||
Args:
|
||||
expression: Expression to qualify
|
||||
|
|
@ -49,6 +41,13 @@ def qualify_tables(
|
|||
|
||||
for scope in traverse_scope(expression):
|
||||
for derived_table in itertools.chain(scope.ctes, scope.derived_tables):
|
||||
if isinstance(derived_table, exp.Subquery):
|
||||
unnested = derived_table.unnest()
|
||||
if isinstance(unnested, exp.Table):
|
||||
joins = unnested.args.pop("joins", None)
|
||||
derived_table.this.replace(exp.select("*").from_(unnested.copy(), copy=False))
|
||||
derived_table.this.set("joins", joins)
|
||||
|
||||
if not derived_table.args.get("alias"):
|
||||
alias_ = next_alias_name()
|
||||
derived_table.set("alias", exp.TableAlias(this=exp.to_identifier(alias_)))
|
||||
|
|
@ -66,19 +65,9 @@ def qualify_tables(
|
|||
if not source.args.get("catalog"):
|
||||
source.set("catalog", exp.to_identifier(catalog))
|
||||
|
||||
# Unnest joins attached in tables by appending them to the closest query
|
||||
for join in source.args.get("joins") or []:
|
||||
scope.expression.append("joins", join)
|
||||
|
||||
source.set("joins", None)
|
||||
source.set("wrapped", None)
|
||||
|
||||
if not source.alias:
|
||||
source = source.replace(
|
||||
alias(
|
||||
source, name or source.name or next_alias_name(), copy=True, table=True
|
||||
)
|
||||
)
|
||||
# Mutates the source by attaching an alias to it
|
||||
alias(source, name or source.name or next_alias_name(), copy=False, table=True)
|
||||
|
||||
pivots = source.args.get("pivots")
|
||||
if pivots and not pivots[0].alias:
|
||||
|
|
|
|||
|
|
@ -122,7 +122,11 @@ class Scope:
|
|||
self._udtfs.append(node)
|
||||
elif isinstance(node, exp.CTE):
|
||||
self._ctes.append(node)
|
||||
elif isinstance(node, exp.Subquery) and isinstance(parent, (exp.From, exp.Join)):
|
||||
elif (
|
||||
isinstance(node, exp.Subquery)
|
||||
and isinstance(parent, (exp.From, exp.Join))
|
||||
and _is_subquery_scope(node)
|
||||
):
|
||||
self._derived_tables.append(node)
|
||||
elif isinstance(node, exp.Subqueryable):
|
||||
self._subqueries.append(node)
|
||||
|
|
@ -274,6 +278,7 @@ class Scope:
|
|||
not ancestor
|
||||
or column.table
|
||||
or isinstance(ancestor, exp.Select)
|
||||
or (isinstance(ancestor, exp.Table) and not isinstance(ancestor.this, exp.Func))
|
||||
or (
|
||||
isinstance(ancestor, exp.Order)
|
||||
and (
|
||||
|
|
@ -340,23 +345,6 @@ class Scope:
|
|||
if isinstance(scope, Scope) and scope.is_cte
|
||||
}
|
||||
|
||||
@property
|
||||
def selects(self):
|
||||
"""
|
||||
Select expressions of this scope.
|
||||
|
||||
For example, for the following expression:
|
||||
SELECT 1 as a, 2 as b FROM x
|
||||
|
||||
The outputs are the "1 as a" and "2 as b" expressions.
|
||||
|
||||
Returns:
|
||||
list[exp.Expression]: expressions
|
||||
"""
|
||||
if isinstance(self.expression, exp.Union):
|
||||
return self.expression.unnest().selects
|
||||
return self.expression.selects
|
||||
|
||||
@property
|
||||
def external_columns(self):
|
||||
"""
|
||||
|
|
@ -548,6 +536,8 @@ def _traverse_scope(scope):
|
|||
yield from _traverse_union(scope)
|
||||
elif isinstance(scope.expression, exp.Subquery):
|
||||
yield from _traverse_subqueries(scope)
|
||||
elif isinstance(scope.expression, exp.Table):
|
||||
yield from _traverse_tables(scope)
|
||||
elif isinstance(scope.expression, exp.UDTF):
|
||||
pass
|
||||
else:
|
||||
|
|
@ -620,6 +610,15 @@ def _traverse_ctes(scope):
|
|||
scope.sources.update(sources)
|
||||
|
||||
|
||||
def _is_subquery_scope(expression: exp.Subquery) -> bool:
|
||||
"""
|
||||
We represent (tbl1 JOIN tbl2) as a Subquery, but it's not really a new scope.
|
||||
If an alias is present, it shadows all names under the Subquery, so that's an
|
||||
exception to this rule.
|
||||
"""
|
||||
return bool(not isinstance(expression.unnest(), exp.Table) or expression.alias)
|
||||
|
||||
|
||||
def _traverse_tables(scope):
|
||||
sources = {}
|
||||
|
||||
|
|
@ -629,9 +628,8 @@ def _traverse_tables(scope):
|
|||
if from_:
|
||||
expressions.append(from_.this)
|
||||
|
||||
for expression in (scope.expression, *scope.find_all(exp.Table)):
|
||||
for join in expression.args.get("joins") or []:
|
||||
expressions.append(join.this)
|
||||
for join in scope.expression.args.get("joins") or []:
|
||||
expressions.append(join.this)
|
||||
|
||||
if isinstance(scope.expression, exp.Table):
|
||||
expressions.append(scope.expression)
|
||||
|
|
@ -655,6 +653,8 @@ def _traverse_tables(scope):
|
|||
sources[find_new_name(sources, table_name)] = expression
|
||||
else:
|
||||
sources[source_name] = expression
|
||||
|
||||
expressions.extend(join.this for join in expression.args.get("joins") or [])
|
||||
continue
|
||||
|
||||
if not isinstance(expression, exp.DerivedTable):
|
||||
|
|
@ -664,10 +664,15 @@ def _traverse_tables(scope):
|
|||
lateral_sources = sources
|
||||
scope_type = ScopeType.UDTF
|
||||
scopes = scope.udtf_scopes
|
||||
else:
|
||||
elif _is_subquery_scope(expression):
|
||||
lateral_sources = None
|
||||
scope_type = ScopeType.DERIVED_TABLE
|
||||
scopes = scope.derived_table_scopes
|
||||
else:
|
||||
# Makes sure we check for possible sources in nested table constructs
|
||||
expressions.append(expression.this)
|
||||
expressions.extend(join.this for join in expression.args.get("joins") or [])
|
||||
continue
|
||||
|
||||
for child_scope in _traverse_scope(
|
||||
scope.branch(
|
||||
|
|
@ -728,7 +733,11 @@ def walk_in_scope(expression, bfs=True):
|
|||
continue
|
||||
if (
|
||||
isinstance(node, exp.CTE)
|
||||
or (isinstance(node, exp.Subquery) and isinstance(parent, (exp.From, exp.Join)))
|
||||
or (
|
||||
isinstance(node, exp.Subquery)
|
||||
and isinstance(parent, (exp.From, exp.Join))
|
||||
and _is_subquery_scope(node)
|
||||
)
|
||||
or isinstance(node, exp.UDTF)
|
||||
or isinstance(node, exp.Subqueryable)
|
||||
):
|
||||
|
|
|
|||
|
|
@ -1708,6 +1708,8 @@ class Parser(metaclass=_Parser):
|
|||
self._match(TokenType.TABLE)
|
||||
this = self._parse_table(schema=True)
|
||||
|
||||
returning = self._parse_returning()
|
||||
|
||||
return self.expression(
|
||||
exp.Insert,
|
||||
this=this,
|
||||
|
|
@ -1717,7 +1719,7 @@ class Parser(metaclass=_Parser):
|
|||
and self._parse_conjunction(),
|
||||
expression=self._parse_ddl_select(),
|
||||
conflict=self._parse_on_conflict(),
|
||||
returning=self._parse_returning(),
|
||||
returning=returning or self._parse_returning(),
|
||||
overwrite=overwrite,
|
||||
alternative=alternative,
|
||||
ignore=ignore,
|
||||
|
|
@ -1761,8 +1763,11 @@ class Parser(metaclass=_Parser):
|
|||
def _parse_returning(self) -> t.Optional[exp.Returning]:
|
||||
if not self._match(TokenType.RETURNING):
|
||||
return None
|
||||
|
||||
return self.expression(exp.Returning, expressions=self._parse_csv(self._parse_column))
|
||||
return self.expression(
|
||||
exp.Returning,
|
||||
expressions=self._parse_csv(self._parse_expression),
|
||||
into=self._match(TokenType.INTO) and self._parse_table_part(),
|
||||
)
|
||||
|
||||
def _parse_row(self) -> t.Optional[exp.RowFormatSerdeProperty | exp.RowFormatDelimitedProperty]:
|
||||
if not self._match(TokenType.FORMAT):
|
||||
|
|
@ -1824,25 +1829,30 @@ class Parser(metaclass=_Parser):
|
|||
if not self._match(TokenType.FROM, advance=False):
|
||||
tables = self._parse_csv(self._parse_table) or None
|
||||
|
||||
returning = self._parse_returning()
|
||||
|
||||
return self.expression(
|
||||
exp.Delete,
|
||||
tables=tables,
|
||||
this=self._match(TokenType.FROM) and self._parse_table(joins=True),
|
||||
using=self._match(TokenType.USING) and self._parse_table(joins=True),
|
||||
where=self._parse_where(),
|
||||
returning=self._parse_returning(),
|
||||
returning=returning or self._parse_returning(),
|
||||
limit=self._parse_limit(),
|
||||
)
|
||||
|
||||
def _parse_update(self) -> exp.Update:
|
||||
this = self._parse_table(alias_tokens=self.UPDATE_ALIAS_TOKENS)
|
||||
expressions = self._match(TokenType.SET) and self._parse_csv(self._parse_equality)
|
||||
returning = self._parse_returning()
|
||||
return self.expression(
|
||||
exp.Update,
|
||||
**{ # type: ignore
|
||||
"this": self._parse_table(alias_tokens=self.UPDATE_ALIAS_TOKENS),
|
||||
"expressions": self._match(TokenType.SET) and self._parse_csv(self._parse_equality),
|
||||
"this": this,
|
||||
"expressions": expressions,
|
||||
"from": self._parse_from(joins=True),
|
||||
"where": self._parse_where(),
|
||||
"returning": self._parse_returning(),
|
||||
"returning": returning or self._parse_returning(),
|
||||
"limit": self._parse_limit(),
|
||||
},
|
||||
)
|
||||
|
|
@ -1969,31 +1979,9 @@ class Parser(metaclass=_Parser):
|
|||
|
||||
self._match_r_paren()
|
||||
|
||||
alias = None
|
||||
|
||||
# Ensure "wrapped" tables are not parsed as Subqueries. The exception to this is when there's
|
||||
# an alias that can be applied to the parentheses, because that would shadow all wrapped table
|
||||
# names, and so we want to parse it as a Subquery to represent the inner scope appropriately.
|
||||
# Additionally, we want the node under the Subquery to be an actual query, so we will replace
|
||||
# the table reference with a star query that selects from it.
|
||||
if isinstance(this, exp.Table):
|
||||
alias = self._parse_table_alias()
|
||||
if not alias:
|
||||
this.set("wrapped", True)
|
||||
return this
|
||||
|
||||
this.set("wrapped", None)
|
||||
joins = this.args.pop("joins", None)
|
||||
this = this.replace(exp.select("*").from_(this.copy(), copy=False))
|
||||
this.set("joins", joins)
|
||||
|
||||
subquery = self._parse_subquery(this, parse_alias=parse_subquery_alias and not alias)
|
||||
if subquery and alias:
|
||||
subquery.set("alias", alias)
|
||||
|
||||
# We return early here so that the UNION isn't attached to the subquery by the
|
||||
# following call to _parse_set_operations, but instead becomes the parent node
|
||||
return subquery
|
||||
return self._parse_subquery(this, parse_alias=parse_subquery_alias)
|
||||
elif self._match(TokenType.VALUES):
|
||||
this = self.expression(
|
||||
exp.Values,
|
||||
|
|
@ -3086,7 +3074,13 @@ class Parser(metaclass=_Parser):
|
|||
if self._match_pair(TokenType.L_BRACKET, TokenType.R_BRACKET):
|
||||
this = exp.DataType(
|
||||
this=exp.DataType.Type.ARRAY,
|
||||
expressions=[exp.DataType.build(type_token.value, expressions=expressions)],
|
||||
expressions=[
|
||||
exp.DataType(
|
||||
this=exp.DataType.Type[type_token.value],
|
||||
expressions=expressions,
|
||||
nested=nested,
|
||||
)
|
||||
],
|
||||
nested=True,
|
||||
)
|
||||
|
||||
|
|
@ -3147,7 +3141,7 @@ class Parser(metaclass=_Parser):
|
|||
return value
|
||||
|
||||
return exp.DataType(
|
||||
this=exp.DataType.Type[type_token.value.upper()],
|
||||
this=exp.DataType.Type[type_token.value],
|
||||
expressions=expressions,
|
||||
nested=nested,
|
||||
values=values,
|
||||
|
|
|
|||
|
|
@ -52,6 +52,7 @@ class TokenType(AutoName):
|
|||
PARAMETER = auto()
|
||||
SESSION_PARAMETER = auto()
|
||||
DAMP = auto()
|
||||
XOR = auto()
|
||||
|
||||
BLOCK_START = auto()
|
||||
BLOCK_END = auto()
|
||||
|
|
@ -590,6 +591,7 @@ class Tokenizer(metaclass=_Tokenizer):
|
|||
"OFFSET": TokenType.OFFSET,
|
||||
"ON": TokenType.ON,
|
||||
"OR": TokenType.OR,
|
||||
"XOR": TokenType.XOR,
|
||||
"ORDER BY": TokenType.ORDER_BY,
|
||||
"ORDINALITY": TokenType.ORDINALITY,
|
||||
"OUTER": TokenType.OUTER,
|
||||
|
|
|
|||
|
|
@ -1356,7 +1356,7 @@ class TestDialect(Validator):
|
|||
write={
|
||||
"duckdb": "CREATE TABLE t (c TEXT, nc TEXT, v1 TEXT, v2 TEXT, nv TEXT, nv2 TEXT)",
|
||||
"hive": "CREATE TABLE t (c CHAR, nc CHAR, v1 STRING, v2 STRING, nv STRING, nv2 STRING)",
|
||||
"oracle": "CREATE TABLE t (c CHAR, nc CHAR, v1 VARCHAR2, v2 VARCHAR2, nv NVARCHAR2, nv2 NVARCHAR2)",
|
||||
"oracle": "CREATE TABLE t (c CHAR, nc NCHAR, v1 VARCHAR2, v2 VARCHAR2, nv NVARCHAR2, nv2 NVARCHAR2)",
|
||||
"postgres": "CREATE TABLE t (c CHAR, nc CHAR, v1 VARCHAR, v2 VARCHAR, nv VARCHAR, nv2 VARCHAR)",
|
||||
"sqlite": "CREATE TABLE t (c TEXT, nc TEXT, v1 TEXT, v2 TEXT, nv TEXT, nv2 TEXT)",
|
||||
},
|
||||
|
|
|
|||
|
|
@ -69,6 +69,8 @@ class TestMySQL(Validator):
|
|||
)
|
||||
|
||||
def test_identity(self):
|
||||
self.validate_identity("SELECT 1 XOR 0")
|
||||
self.validate_identity("SELECT 1 && 0", "SELECT 1 AND 0")
|
||||
self.validate_identity("SELECT /*+ BKA(t1) NO_BKA(t2) */ * FROM t1 INNER JOIN t2")
|
||||
self.validate_identity("SELECT /*+ MERGE(dt) */ * FROM (SELECT * FROM t1) AS dt")
|
||||
self.validate_identity("SELECT /*+ INDEX(t, i) */ c1 FROM t WHERE c2 = 'value'")
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
from sqlglot.errors import UnsupportedError
|
||||
from tests.dialects.test_dialect import Validator
|
||||
|
||||
|
||||
|
|
@ -57,8 +58,17 @@ class TestOracle(Validator):
|
|||
|
||||
def test_join_marker(self):
|
||||
self.validate_identity("SELECT e1.x, e2.x FROM e e1, e e2 WHERE e1.y (+) = e2.y")
|
||||
self.validate_identity("SELECT e1.x, e2.x FROM e e1, e e2 WHERE e1.y = e2.y (+)")
|
||||
self.validate_identity("SELECT e1.x, e2.x FROM e e1, e e2 WHERE e1.y (+) = e2.y (+)")
|
||||
|
||||
self.validate_all(
|
||||
"SELECT e1.x, e2.x FROM e e1, e e2 WHERE e1.y = e2.y (+)", write={"": UnsupportedError}
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT e1.x, e2.x FROM e e1, e e2 WHERE e1.y = e2.y (+)",
|
||||
write={
|
||||
"": "SELECT e1.x, e2.x FROM e AS e1, e AS e2 WHERE e1.y = e2.y",
|
||||
"oracle": "SELECT e1.x, e2.x FROM e e1, e e2 WHERE e1.y = e2.y (+)",
|
||||
},
|
||||
)
|
||||
|
||||
def test_hints(self):
|
||||
self.validate_identity("SELECT /*+ USE_NL(A B) */ A.COL_TEST FROM TABLE_A A, TABLE_B B")
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ class TestPostgres(Validator):
|
|||
dialect = "postgres"
|
||||
|
||||
def test_ddl(self):
|
||||
self.validate_identity("CREATE TABLE test (elems JSONB[])")
|
||||
self.validate_identity("CREATE TABLE public.y (x TSTZRANGE NOT NULL)")
|
||||
self.validate_identity("CREATE TABLE test (foo HSTORE)")
|
||||
self.validate_identity("CREATE TABLE test (foo JSONB)")
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
from unittest import mock
|
||||
|
||||
from sqlglot import UnsupportedError, exp, parse_one
|
||||
from tests.dialects.test_dialect import Validator
|
||||
|
||||
|
|
@ -309,6 +311,7 @@ class TestSnowflake(Validator):
|
|||
"SELECT IFF(TRUE, 'true', 'false')",
|
||||
write={
|
||||
"snowflake": "SELECT IFF(TRUE, 'true', 'false')",
|
||||
"spark": "SELECT IF(TRUE, 'true', 'false')",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
|
|
@ -870,6 +873,46 @@ FROM persons AS p, LATERAL FLATTEN(input => p.c, path => 'contact') AS f, LATERA
|
|||
self.assertIsInstance(ilike, exp.ILikeAny)
|
||||
like.sql() # check that this doesn't raise
|
||||
|
||||
@mock.patch("sqlglot.generator.logger")
|
||||
def test_regexp_substr(self, logger):
|
||||
self.validate_all(
|
||||
"REGEXP_SUBSTR(subject, pattern, pos, occ, params, group)",
|
||||
write={
|
||||
"bigquery": "REGEXP_EXTRACT(subject, pattern, pos, occ)",
|
||||
"hive": "REGEXP_EXTRACT(subject, pattern, group)",
|
||||
"presto": "REGEXP_EXTRACT(subject, pattern, group)",
|
||||
"snowflake": "REGEXP_SUBSTR(subject, pattern, pos, occ, params, group)",
|
||||
"spark": "REGEXP_EXTRACT(subject, pattern, group)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"REGEXP_SUBSTR(subject, pattern)",
|
||||
read={
|
||||
"bigquery": "REGEXP_EXTRACT(subject, pattern)",
|
||||
"hive": "REGEXP_EXTRACT(subject, pattern)",
|
||||
"presto": "REGEXP_EXTRACT(subject, pattern)",
|
||||
"spark": "REGEXP_EXTRACT(subject, pattern)",
|
||||
},
|
||||
write={
|
||||
"bigquery": "REGEXP_EXTRACT(subject, pattern)",
|
||||
"hive": "REGEXP_EXTRACT(subject, pattern)",
|
||||
"presto": "REGEXP_EXTRACT(subject, pattern)",
|
||||
"snowflake": "REGEXP_SUBSTR(subject, pattern)",
|
||||
"spark": "REGEXP_EXTRACT(subject, pattern)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"REGEXP_SUBSTR(subject, pattern, 1, 1, 'c', group)",
|
||||
read={
|
||||
"bigquery": "REGEXP_SUBSTR(subject, pattern, 1, 1, 'c', group)",
|
||||
"duckdb": "REGEXP_EXTRACT(subject, pattern, group)",
|
||||
"hive": "REGEXP_EXTRACT(subject, pattern, group)",
|
||||
"presto": "REGEXP_EXTRACT(subject, pattern, group)",
|
||||
"snowflake": "REGEXP_SUBSTR(subject, pattern, 1, 1, 'c', group)",
|
||||
"spark": "REGEXP_EXTRACT(subject, pattern, group)",
|
||||
},
|
||||
)
|
||||
|
||||
def test_match_recognize(self):
|
||||
for row in (
|
||||
"ONE ROW PER MATCH",
|
||||
|
|
|
|||
|
|
@ -6,6 +6,11 @@ class TestTSQL(Validator):
|
|||
dialect = "tsql"
|
||||
|
||||
def test_tsql(self):
|
||||
self.validate_identity("UPDATE x SET y = 1 OUTPUT x.a, x.b INTO @y FROM y")
|
||||
self.validate_identity("UPDATE x SET y = 1 OUTPUT x.a, x.b FROM y")
|
||||
self.validate_identity("INSERT INTO x (y) OUTPUT x.a, x.b INTO l SELECT * FROM z")
|
||||
self.validate_identity("INSERT INTO x (y) OUTPUT x.a, x.b SELECT * FROM z")
|
||||
self.validate_identity("DELETE x OUTPUT x.a FROM z")
|
||||
self.validate_identity("SELECT * FROM t WITH (TABLOCK, INDEX(myindex))")
|
||||
self.validate_identity("SELECT * FROM t WITH (NOWAIT)")
|
||||
self.validate_identity("SELECT CASE WHEN a > 1 THEN b END")
|
||||
|
|
@ -205,6 +210,30 @@ class TestTSQL(Validator):
|
|||
},
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"CAST(x as FLOAT(32))",
|
||||
write={"tsql": "CAST(x AS FLOAT(32))", "hive": "CAST(x AS FLOAT)"},
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"CAST(x as FLOAT(64))",
|
||||
write={"tsql": "CAST(x AS FLOAT(64))", "spark": "CAST(x AS DOUBLE)"},
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"CAST(x as FLOAT(6))", write={"tsql": "CAST(x AS FLOAT(6))", "hive": "CAST(x AS FLOAT)"}
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"CAST(x as FLOAT(36))",
|
||||
write={"tsql": "CAST(x AS FLOAT(36))", "hive": "CAST(x AS DOUBLE)"},
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"CAST(x as FLOAT(99))",
|
||||
write={"tsql": "CAST(x AS FLOAT(99))", "hive": "CAST(x AS DOUBLE)"},
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"CAST(x as DOUBLE)",
|
||||
write={
|
||||
|
|
|
|||
17
tests/fixtures/identity.sql
vendored
17
tests/fixtures/identity.sql
vendored
|
|
@ -423,6 +423,8 @@ SELECT 1 INTERSECT SELECT 2
|
|||
SELECT 1 INTERSECT SELECT 2
|
||||
SELECT 1 AS delete, 2 AS alter
|
||||
SELECT * FROM (x)
|
||||
SELECT * FROM ((x))
|
||||
SELECT * FROM (((x)))
|
||||
SELECT * FROM ((SELECT 1))
|
||||
SELECT * FROM (x CROSS JOIN foo LATERAL VIEW EXPLODE(y))
|
||||
SELECT * FROM (SELECT 1) AS x
|
||||
|
|
@ -432,6 +434,14 @@ SELECT * FROM (SELECT 1 UNION ALL SELECT 2)
|
|||
SELECT * FROM ((SELECT 1) AS a UNION ALL (SELECT 2) AS b)
|
||||
SELECT * FROM ((SELECT 1) AS a(b))
|
||||
SELECT * FROM ((SELECT 1) UNION (SELECT 2) UNION (SELECT 3))
|
||||
SELECT * FROM (table1 AS t1 LEFT JOIN table2 AS t2 ON 1 = 1)
|
||||
SELECT * FROM (tbl1 LEFT JOIN tbl2 ON 1 = 1)
|
||||
SELECT * FROM (tbl1, tbl2 JOIN tbl3 ON TRUE)
|
||||
SELECT * FROM (tbl1 CROSS JOIN tbl2)
|
||||
SELECT * FROM (tbl1 CROSS JOIN tbl2) AS t
|
||||
SELECT * FROM (tbl AS tbl) AS t
|
||||
SELECT * FROM (tbl1 JOIN (tbl2 CROSS JOIN tbl3) ON bla = foo)
|
||||
SELECT * FROM (tbl1, LATERAL (SELECT * FROM bla) AS tbl)
|
||||
SELECT * FROM x AS y(a, b)
|
||||
SELECT * EXCEPT (a, b)
|
||||
SELECT * EXCEPT (a, b) FROM y
|
||||
|
|
@ -607,6 +617,7 @@ CREATE FUNCTION a() LANGUAGE sql RETURNS INT
|
|||
CREATE FUNCTION a.b(x INT) RETURNS INT AS RETURN x + 1
|
||||
CREATE FUNCTION a.b.c()
|
||||
CREATE INDEX abc ON t (a)
|
||||
CREATE INDEX "abc" ON t (a)
|
||||
CREATE INDEX abc ON t (a, b, b)
|
||||
CREATE INDEX abc ON t (a NULLS LAST)
|
||||
CREATE INDEX pointloc ON points USING GIST(BOX(location, location))
|
||||
|
|
@ -700,6 +711,7 @@ UPDATE tbl_name SET foo = 123
|
|||
UPDATE tbl_name SET foo = 123, bar = 345
|
||||
UPDATE db.tbl_name SET foo = 123 WHERE tbl_name.bar = 234
|
||||
UPDATE db.tbl_name SET foo = 123, foo_1 = 234 WHERE tbl_name.bar = 234
|
||||
UPDATE products SET price = price * 1.10 WHERE price <= 99.99 RETURNING name, price AS new_price
|
||||
TRUNCATE TABLE x
|
||||
OPTIMIZE TABLE y
|
||||
VACUUM FREEZE my_table
|
||||
|
|
@ -721,11 +733,6 @@ WITH a AS ((SELECT 1 AS b) UNION ALL (SELECT 1 AS b)) SELECT * FROM a
|
|||
SELECT (WITH x AS (SELECT 1 AS y) SELECT * FROM x) AS z
|
||||
SELECT ((SELECT 1) + 1)
|
||||
SELECT * FROM project.dataset.INFORMATION_SCHEMA.TABLES
|
||||
SELECT * FROM (table1 AS t1 LEFT JOIN table2 AS t2 ON 1 = 1)
|
||||
SELECT * FROM (tbl1 LEFT JOIN tbl2 ON 1 = 1)
|
||||
SELECT * FROM (tbl1, tbl2 JOIN tbl3 ON TRUE)
|
||||
SELECT * FROM (tbl1 JOIN (tbl2 CROSS JOIN tbl3) ON bla = foo)
|
||||
SELECT * FROM (tbl1, LATERAL (SELECT * FROM bla) AS tbl)
|
||||
SELECT CAST(x AS INT) /* comment */ FROM foo
|
||||
SELECT a /* x */, b /* x */
|
||||
SELECT a /* x */ /* y */ /* z */, b /* k */ /* m */
|
||||
|
|
|
|||
105
tests/fixtures/optimizer/optimizer.sql
vendored
105
tests/fixtures/optimizer/optimizer.sql
vendored
|
|
@ -701,3 +701,108 @@ SELECT
|
|||
"x"."a" * "x"."b" - "x"."b"
|
||||
) AS "f"
|
||||
FROM "x" AS "x";
|
||||
|
||||
# title: wrapped table without alias
|
||||
# execute: false
|
||||
SELECT * FROM (tbl);
|
||||
SELECT
|
||||
*
|
||||
FROM (
|
||||
"tbl" AS "tbl"
|
||||
);
|
||||
|
||||
# title: wrapped table with alias
|
||||
# execute: false
|
||||
SELECT * FROM (tbl AS tbl);
|
||||
SELECT
|
||||
*
|
||||
FROM (
|
||||
"tbl" AS "tbl"
|
||||
);
|
||||
|
||||
# title: wrapped join of tables without alias
|
||||
SELECT a, c FROM (x LEFT JOIN y ON a = c);
|
||||
SELECT
|
||||
"x"."a" AS "a",
|
||||
"y"."c" AS "c"
|
||||
FROM (
|
||||
"x" AS "x"
|
||||
LEFT JOIN "y" AS "y"
|
||||
ON "x"."a" = "y"."c"
|
||||
);
|
||||
|
||||
# title: wrapped join of tables with alias
|
||||
# execute: false
|
||||
SELECT a, c FROM (x LEFT JOIN y ON a = c) AS t;
|
||||
SELECT
|
||||
"x"."a" AS "a",
|
||||
"y"."c" AS "c"
|
||||
FROM "x" AS "x"
|
||||
LEFT JOIN "y" AS "y"
|
||||
ON "x"."a" = "y"."c";
|
||||
|
||||
# title: chained wrapped joins without aliases
|
||||
# execute: false
|
||||
SELECT * FROM ((a CROSS JOIN ((b CROSS JOIN c) CROSS JOIN (d CROSS JOIN e))));
|
||||
SELECT
|
||||
*
|
||||
FROM (
|
||||
(
|
||||
"a" AS "a"
|
||||
CROSS JOIN (
|
||||
(
|
||||
"b" AS "b"
|
||||
CROSS JOIN "c" AS "c"
|
||||
)
|
||||
CROSS JOIN (
|
||||
"d" AS "d"
|
||||
CROSS JOIN "e" AS "e"
|
||||
)
|
||||
)
|
||||
)
|
||||
);
|
||||
|
||||
# title: chained wrapped joins with aliases
|
||||
# execute: false
|
||||
SELECT * FROM ((a AS foo CROSS JOIN b AS bar) CROSS JOIN c AS baz);
|
||||
SELECT
|
||||
*
|
||||
FROM (
|
||||
(
|
||||
"a" AS "foo"
|
||||
CROSS JOIN "b" AS "bar"
|
||||
)
|
||||
CROSS JOIN "c" AS "baz"
|
||||
);
|
||||
|
||||
# title: table joined with join construct
|
||||
SELECT x.a, y.b, z.c FROM x LEFT JOIN (y INNER JOIN z ON y.c = z.c) ON x.b = y.b;
|
||||
SELECT
|
||||
"x"."a" AS "a",
|
||||
"y"."b" AS "b",
|
||||
"z"."c" AS "c"
|
||||
FROM "x" AS "x"
|
||||
LEFT JOIN (
|
||||
"y" AS "y"
|
||||
JOIN "z" AS "z"
|
||||
ON "y"."c" = "z"."c"
|
||||
)
|
||||
ON "x"."b" = "y"."b";
|
||||
|
||||
# title: select * from table joined with join construct
|
||||
# execute: false
|
||||
SELECT * FROM x LEFT JOIN (y INNER JOIN z ON y.c = z.c) ON x.b = y.b;
|
||||
SELECT
|
||||
"y"."b" AS "b",
|
||||
"y"."c" AS "c",
|
||||
"z"."a" AS "a",
|
||||
"z"."c" AS "c",
|
||||
"x"."a" AS "a",
|
||||
"x"."b" AS "b"
|
||||
FROM "x" AS "x"
|
||||
LEFT JOIN (
|
||||
"y" AS "y"
|
||||
JOIN "z" AS "z"
|
||||
ON "y"."c" = "z"."c"
|
||||
)
|
||||
ON "x"."b" = "y"."b";
|
||||
|
|
|
|||
4
tests/fixtures/optimizer/qualify_columns.sql
vendored
4
tests/fixtures/optimizer/qualify_columns.sql
vendored
|
|
@ -430,6 +430,10 @@ SELECT x.a AS a, x.b AS b FROM x AS x QUALIFY COUNT(x.a) OVER (PARTITION BY x.b)
|
|||
-- Expand laterals
|
||||
--------------------------------------
|
||||
|
||||
# execute: false
|
||||
select 2 AS d, d + 1 FROM x WHERE d = 2 GROUP BY d;
|
||||
SELECT 2 AS d, 2 + 1 AS _col_1 FROM x AS x WHERE 2 = 2 GROUP BY 1;
|
||||
|
||||
# title: expand alias reference
|
||||
SELECT
|
||||
x.a + 1 AS i,
|
||||
|
|
|
|||
50
tests/fixtures/optimizer/qualify_tables.sql
vendored
50
tests/fixtures/optimizer/qualify_tables.sql
vendored
|
|
@ -26,45 +26,49 @@ SELECT (SELECT y.c FROM c.db.y AS y) FROM c.db.x AS x;
|
|||
SELECT * FROM x PIVOT (SUM(a) FOR b IN ('a', 'b'));
|
||||
SELECT * FROM c.db.x AS x PIVOT(SUM(a) FOR b IN ('a', 'b')) AS _q_0;
|
||||
|
||||
-----------------------------------------------------------
|
||||
--- Unnest wrapped tables / joins, expand join constructs
|
||||
-----------------------------------------------------------
|
||||
|
||||
# title: wrapped table without alias
|
||||
SELECT * FROM (tbl);
|
||||
SELECT * FROM c.db.tbl AS tbl;
|
||||
SELECT * FROM (c.db.tbl AS tbl);
|
||||
|
||||
# title: wrapped table with alias
|
||||
SELECT * FROM (tbl AS tbl);
|
||||
SELECT * FROM c.db.tbl AS tbl;
|
||||
SELECT * FROM (c.db.tbl AS tbl);
|
||||
|
||||
# title: wrapped table with alias and multiple redundant parentheses
|
||||
# title: wrapped table with alias using multiple (redundant) parentheses
|
||||
SELECT * FROM ((((tbl AS tbl))));
|
||||
SELECT * FROM c.db.tbl AS tbl;
|
||||
SELECT * FROM ((((c.db.tbl AS tbl))));
|
||||
|
||||
# title: wrapped join of tables without alias
|
||||
SELECT * FROM (t1 CROSS JOIN t2);
|
||||
SELECT * FROM (c.db.t1 AS t1 CROSS JOIN c.db.t2 AS t2);
|
||||
|
||||
# title: wrapped join of tables with alias, expansion of join construct
|
||||
SELECT * FROM (t1 CROSS JOIN t2) AS t;
|
||||
SELECT * FROM (SELECT * FROM c.db.t1 AS t1 CROSS JOIN c.db.t2 AS t2) AS t;
|
||||
|
||||
# title: chained wrapped joins without aliases (1)
|
||||
SELECT * FROM ((a CROSS JOIN b) CROSS JOIN c);
|
||||
SELECT * FROM c.db.a AS a CROSS JOIN c.db.b AS b CROSS JOIN c.db.c AS c;
|
||||
SELECT * FROM ((c.db.a AS a CROSS JOIN c.db.b AS b) CROSS JOIN c.db.c AS c);
|
||||
|
||||
# title: chained wrapped joins without aliases (2)
|
||||
SELECT * FROM (a CROSS JOIN (b CROSS JOIN c));
|
||||
SELECT * FROM c.db.a AS a CROSS JOIN c.db.b AS b CROSS JOIN c.db.c AS c;
|
||||
SELECT * FROM (c.db.a AS a CROSS JOIN (c.db.b AS b CROSS JOIN c.db.c AS c));
|
||||
|
||||
# title: chained wrapped joins without aliases (3)
|
||||
SELECT * FROM ((a CROSS JOIN ((b CROSS JOIN c) CROSS JOIN d)));
|
||||
SELECT * FROM c.db.a AS a CROSS JOIN c.db.b AS b CROSS JOIN c.db.c AS c CROSS JOIN c.db.d AS d;
|
||||
SELECT * FROM ((c.db.a AS a CROSS JOIN ((c.db.b AS b CROSS JOIN c.db.c AS c) CROSS JOIN c.db.d AS d)));
|
||||
|
||||
# title: chained wrapped joins without aliases (4)
|
||||
SELECT * FROM ((a CROSS JOIN ((b CROSS JOIN c) CROSS JOIN (d CROSS JOIN e))));
|
||||
SELECT * FROM c.db.a AS a CROSS JOIN c.db.b AS b CROSS JOIN c.db.c AS c CROSS JOIN c.db.d AS d CROSS JOIN c.db.e AS e;
|
||||
SELECT * FROM ((c.db.a AS a CROSS JOIN ((c.db.b AS b CROSS JOIN c.db.c AS c) CROSS JOIN (c.db.d AS d CROSS JOIN c.db.e AS e))));
|
||||
|
||||
# title: chained wrapped joins with aliases
|
||||
SELECT * FROM ((a AS foo CROSS JOIN b AS bar) CROSS JOIN c AS baz);
|
||||
SELECT * FROM c.db.a AS foo CROSS JOIN c.db.b AS bar CROSS JOIN c.db.c AS baz;
|
||||
SELECT * FROM ((c.db.a AS foo CROSS JOIN c.db.b AS bar) CROSS JOIN c.db.c AS baz);
|
||||
|
||||
# title: wrapped join with subquery without alias
|
||||
SELECT * FROM (tbl1 CROSS JOIN (SELECT * FROM tbl2) AS t1);
|
||||
SELECT * FROM c.db.tbl1 AS tbl1 CROSS JOIN (SELECT * FROM c.db.tbl2 AS tbl2) AS t1;
|
||||
SELECT * FROM (c.db.tbl1 AS tbl1 CROSS JOIN (SELECT * FROM c.db.tbl2 AS tbl2) AS t1);
|
||||
|
||||
# title: wrapped join with subquery with alias, parentheses can't be omitted because of alias
|
||||
SELECT * FROM (tbl1 CROSS JOIN (SELECT * FROM tbl2) AS t1) AS t2;
|
||||
|
|
@ -72,32 +76,28 @@ SELECT * FROM (SELECT * FROM c.db.tbl1 AS tbl1 CROSS JOIN (SELECT * FROM c.db.tb
|
|||
|
||||
# title: join construct as the right operand of a left join
|
||||
SELECT * FROM a LEFT JOIN (b INNER JOIN c ON c.id = b.id) ON b.id = a.id;
|
||||
SELECT * FROM c.db.a AS a LEFT JOIN c.db.b AS b ON b.id = a.id INNER JOIN c.db.c AS c ON c.id = b.id;
|
||||
SELECT * FROM c.db.a AS a LEFT JOIN (c.db.b AS b INNER JOIN c.db.c AS c ON c.id = b.id) ON b.id = a.id;
|
||||
|
||||
# title: nested joins converted to canonical form
|
||||
# title: nested joins
|
||||
SELECT * FROM a LEFT JOIN b INNER JOIN c ON c.id = b.id ON b.id = a.id;
|
||||
SELECT * FROM c.db.a AS a LEFT JOIN c.db.b AS b ON b.id = a.id INNER JOIN c.db.c AS c ON c.id = b.id;
|
||||
SELECT * FROM c.db.a AS a LEFT JOIN c.db.b AS b INNER JOIN c.db.c AS c ON c.id = b.id ON b.id = a.id;
|
||||
|
||||
# title: parentheses can't be omitted because alias shadows inner table names
|
||||
SELECT t.a FROM (tbl AS tbl) AS t;
|
||||
SELECT t.a FROM (SELECT * FROM c.db.tbl AS tbl) AS t;
|
||||
|
||||
# title: outermost set of parentheses can't be omitted due to shadowing (1)
|
||||
SELECT * FROM ((tbl AS tbl)) AS _q_0;
|
||||
SELECT * FROM (SELECT * FROM c.db.tbl AS tbl) AS _q_0;
|
||||
|
||||
# title: outermost set of parentheses can't be omitted due to shadowing (2)
|
||||
# title: wrapped aliased table with outer alias
|
||||
SELECT * FROM ((((tbl AS tbl)))) AS _q_0;
|
||||
SELECT * FROM (SELECT * FROM c.db.tbl AS tbl) AS _q_0;
|
||||
|
||||
# title: join construct with three tables in canonical form
|
||||
# title: join construct with three tables
|
||||
SELECT * FROM (tbl1 AS tbl1 JOIN tbl2 AS tbl2 ON id1 = id2 JOIN tbl3 AS tbl3 ON id1 = id3) AS _q_0;
|
||||
SELECT * FROM (SELECT * FROM c.db.tbl1 AS tbl1 JOIN c.db.tbl2 AS tbl2 ON id1 = id2 JOIN c.db.tbl3 AS tbl3 ON id1 = id3) AS _q_0;
|
||||
|
||||
# title: join construct with three tables in canonical form and redundant set of parentheses
|
||||
# title: join construct with three tables and redundant set of parentheses
|
||||
SELECT * FROM ((tbl1 AS tbl1 JOIN tbl2 AS tbl2 ON id1 = id2 JOIN tbl3 AS tbl3 ON id1 = id3)) AS _q_0;
|
||||
SELECT * FROM (SELECT * FROM c.db.tbl1 AS tbl1 JOIN c.db.tbl2 AS tbl2 ON id1 = id2 JOIN c.db.tbl3 AS tbl3 ON id1 = id3) AS _q_0;
|
||||
|
||||
# title: nested join construct in canonical form
|
||||
# title: join construct within join construct
|
||||
SELECT * FROM (tbl1 AS tbl1 JOIN (tbl2 AS tbl2 JOIN tbl3 AS tbl3 ON id2 = id3) AS _q_0 ON id1 = id3) AS _q_1;
|
||||
SELECT * FROM (SELECT * FROM c.db.tbl1 AS tbl1 JOIN (SELECT * FROM c.db.tbl2 AS tbl2 JOIN c.db.tbl3 AS tbl3 ON id2 = id3) AS _q_0 ON id1 = id3) AS _q_1;
|
||||
|
|
|
|||
|
|
@ -183,6 +183,8 @@ class TestExpressions(unittest.TestCase):
|
|||
exp.table_name(parse_one("foo.`{bar,er}`", read="databricks"), dialect="databricks"),
|
||||
"foo.`{bar,er}`",
|
||||
)
|
||||
self.assertEqual(exp.table_name(exp.to_table("a-1.b.c", dialect="bigquery")), '"a-1".b.c')
|
||||
self.assertEqual(exp.table_name(exp.to_table("a.b.c.d.e", dialect="bigquery")), "a.b.c.d.e")
|
||||
|
||||
def test_table(self):
|
||||
self.assertEqual(exp.table_("a", alias="b"), parse_one("select * from a b").find(exp.Table))
|
||||
|
|
|
|||
|
|
@ -571,7 +571,7 @@ class TestParser(unittest.TestCase):
|
|||
"""
|
||||
)
|
||||
self.assertIsNotNone(query)
|
||||
self.assertLessEqual(time.time() - now, 0.1)
|
||||
self.assertLessEqual(time.time() - now, 0.2)
|
||||
|
||||
def test_parse_properties(self):
|
||||
self.assertEqual(
|
||||
|
|
@ -580,58 +580,3 @@ class TestParser(unittest.TestCase):
|
|||
|
||||
def test_parse_floats(self):
|
||||
self.assertTrue(parse_one("1. ").is_number)
|
||||
|
||||
def test_parse_wrapped_tables(self):
|
||||
expr = parse_one("select * from (table)")
|
||||
self.assertIsInstance(expr.args["from"].this, exp.Table)
|
||||
self.assertTrue(expr.args["from"].this.args["wrapped"])
|
||||
|
||||
expr = parse_one("select * from (((table)))")
|
||||
self.assertIsInstance(expr.args["from"].this, exp.Table)
|
||||
self.assertTrue(expr.args["from"].this.args["wrapped"])
|
||||
|
||||
self.assertEqual(expr.sql(), "SELECT * FROM (table)")
|
||||
|
||||
expr = parse_one("select * from (tbl1 join tbl2)")
|
||||
self.assertIsInstance(expr.args["from"].this, exp.Table)
|
||||
self.assertTrue(expr.args["from"].this.args["wrapped"])
|
||||
self.assertEqual(len(expr.args["from"].this.args["joins"]), 1)
|
||||
|
||||
expr = parse_one("select * from (tbl1 join tbl2) t")
|
||||
self.assertIsInstance(expr.args["from"].this, exp.Subquery)
|
||||
self.assertIsInstance(expr.args["from"].this.this, exp.Select)
|
||||
self.assertEqual(expr.sql(), "SELECT * FROM (SELECT * FROM tbl1, tbl2) AS t")
|
||||
|
||||
expr = parse_one("select * from (tbl as tbl) t")
|
||||
self.assertEqual(expr.sql(), "SELECT * FROM (SELECT * FROM tbl AS tbl) AS t")
|
||||
|
||||
expr = parse_one("select * from ((a cross join b) cross join c)")
|
||||
self.assertIsInstance(expr.args["from"].this, exp.Table)
|
||||
self.assertTrue(expr.args["from"].this.args["wrapped"])
|
||||
self.assertEqual(len(expr.args["from"].this.args["joins"]), 2)
|
||||
self.assertEqual(expr.sql(), "SELECT * FROM (a CROSS JOIN b CROSS JOIN c)")
|
||||
|
||||
expr = parse_one("select * from ((a cross join b) cross join c) t")
|
||||
self.assertIsInstance(expr.args["from"].this, exp.Subquery)
|
||||
self.assertEqual(len(expr.args["from"].this.this.args["joins"]), 2)
|
||||
self.assertEqual(
|
||||
expr.sql(), "SELECT * FROM (SELECT * FROM a CROSS JOIN b CROSS JOIN c) AS t"
|
||||
)
|
||||
|
||||
expr = parse_one("select * from (a cross join (b cross join c))")
|
||||
self.assertIsInstance(expr.args["from"].this, exp.Table)
|
||||
self.assertTrue(expr.args["from"].this.args["wrapped"])
|
||||
self.assertEqual(len(expr.args["from"].this.args["joins"]), 1)
|
||||
self.assertIsInstance(expr.args["from"].this.args["joins"][0].this, exp.Table)
|
||||
self.assertTrue(expr.args["from"].this.args["joins"][0].this.args["wrapped"])
|
||||
self.assertEqual(expr.sql(), "SELECT * FROM (a CROSS JOIN (b CROSS JOIN c))")
|
||||
|
||||
expr = parse_one("select * from ((a cross join ((b cross join c) cross join d)))")
|
||||
self.assertEqual(expr.sql(), "SELECT * FROM (a CROSS JOIN (b CROSS JOIN c CROSS JOIN d))")
|
||||
|
||||
expr = parse_one(
|
||||
"select * from ((a cross join ((b cross join c) cross join (d cross join e))))"
|
||||
)
|
||||
self.assertEqual(
|
||||
expr.sql(), "SELECT * FROM (a CROSS JOIN (b CROSS JOIN c CROSS JOIN (d CROSS JOIN e)))"
|
||||
)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue