Adding upstream version 16.4.0.
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
parent
d61627452f
commit
cac8fd11fe
GPG key ID:
FBB4F0E80A80222F
90 changed files with 35638 additions and 33343 deletions
51
CHANGELOG.md
51
CHANGELOG.md
|
|
@ -1,6 +1,54 @@
|
|||
Changelog
|
||||
=========
|
||||
|
||||
## [v16.3.1] - 2023-06-16
|
||||
### :bug: Bug Fixes
|
||||
- [`18db68c`](https://github.com/tobymao/sqlglot/commit/18db68c15e607884572adaae3dd6bd0c6c4bc582) - cluster/distribute/sort by for hive *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
|
||||
|
||||
## [v16.3.0] - 2023-06-16
|
||||
### :boom: BREAKING CHANGES
|
||||
- due to [`038afc9`](https://github.com/tobymao/sqlglot/commit/038afc90b1f3fe261ea6ffb4d3654006bb4317fd) - switch presto tsords to cast timestamp -> date *(commit by [@tobymao](https://github.com/tobymao))*:
|
||||
|
||||
switch presto tsords to cast timestamp -> date
|
||||
|
||||
- due to [`4084ba3`](https://github.com/tobymao/sqlglot/commit/4084ba322d8ae07620c69bf7586571d209e68917) - move normalization logic in Dialect, update case-sensitivity info *(PR [#1784](https://github.com/tobymao/sqlglot/pull/1784) by [@GeorgeSittas](https://github.com/GeorgeSittas))*:
|
||||
|
||||
move normalization logic in Dialect, update case-sensitivity info (#1784)
|
||||
|
||||
|
||||
### :sparkles: New Features
|
||||
- [`fc9afb3`](https://github.com/tobymao/sqlglot/commit/fc9afb3e3033bdf61d58592625f23c6f915370e0) - **snowflake**: add support for COPY GRANTS property *(PR [#1793](https://github.com/tobymao/sqlglot/pull/1793) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- :arrow_lower_right: *addresses issue [#1791](undefined) opened by [@vedas2](https://github.com/vedas2)*
|
||||
|
||||
### :bug: Bug Fixes
|
||||
- [`311380c`](https://github.com/tobymao/sqlglot/commit/311380c14b5f9627e0b5fea5a04e60d23c062dd9) - select as struct transpilation closes [#1788](https://github.com/tobymao/sqlglot/pull/1788) *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
- [`1b62c0a`](https://github.com/tobymao/sqlglot/commit/1b62c0a3a4cbf3d5ac442e7c8fa9e462e3fff982) - **parser**: cast coalesce arg to text in the context of a CONCAT call *(PR [#1792](https://github.com/tobymao/sqlglot/pull/1792) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- [`d27e8f8`](https://github.com/tobymao/sqlglot/commit/d27e8f8ead98373fc5b699abaeb0a235efcf9f6e) - **schema**: ensure tables aren't normalized for BigQuery *(PR [#1794](https://github.com/tobymao/sqlglot/pull/1794) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
- [`58fe190`](https://github.com/tobymao/sqlglot/commit/58fe1903166b88eddb82ceddd296c3363cd8a06d) - rawstring backslashes for bigquery *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
|
||||
### :recycle: Refactors
|
||||
- [`038afc9`](https://github.com/tobymao/sqlglot/commit/038afc90b1f3fe261ea6ffb4d3654006bb4317fd) - switch presto tsords to cast timestamp -> date *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
- [`4084ba3`](https://github.com/tobymao/sqlglot/commit/4084ba322d8ae07620c69bf7586571d209e68917) - move normalization logic in Dialect, update case-sensitivity info *(PR [#1784](https://github.com/tobymao/sqlglot/pull/1784) by [@GeorgeSittas](https://github.com/GeorgeSittas))*
|
||||
|
||||
|
||||
## [v16.2.1] - 2023-06-16
|
||||
### :boom: BREAKING CHANGES
|
||||
- due to [`88249b8`](https://github.com/tobymao/sqlglot/commit/88249b825fbe446e3450a364ef8e108e8495767e) - cyclic joins in the optimizer *(PR [#1786](https://github.com/tobymao/sqlglot/pull/1786) by [@tobymao](https://github.com/tobymao))*:
|
||||
|
||||
cyclic joins in the optimizer (#1786)
|
||||
|
||||
|
||||
### :bug: Bug Fixes
|
||||
- [`88249b8`](https://github.com/tobymao/sqlglot/commit/88249b825fbe446e3450a364ef8e108e8495767e) - cyclic joins in the optimizer *(PR [#1786](https://github.com/tobymao/sqlglot/pull/1786) by [@tobymao](https://github.com/tobymao))*
|
||||
- [`f957a07`](https://github.com/tobymao/sqlglot/commit/f957a07468df3e9a30393cbcb9baafcb41ad0bc6) - overly aggressive cross join removal *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
- [`cacf8bf`](https://github.com/tobymao/sqlglot/commit/cacf8bf65dcb0dfc9c0b6339d84f2cfec9bcb46b) - build null types *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
|
||||
### :wrench: Chores
|
||||
- [`d696d7f`](https://github.com/tobymao/sqlglot/commit/d696d7fe195a841d9418487627496054b0d4492f) - cleanup merge_subqueries *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
- [`8e1b6a7`](https://github.com/tobymao/sqlglot/commit/8e1b6a723fa48181314dcb867a04ff1346bab27b) - speed up executor tests *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
|
||||
|
||||
## [v16.2.0] - 2023-06-15
|
||||
### :bug: Bug Fixes
|
||||
- [`b29a421`](https://github.com/tobymao/sqlglot/commit/b29a421843bc94d88e5f67dd787ee07a675d16ab) - parsing unknown into data type build *(commit by [@tobymao](https://github.com/tobymao))*
|
||||
|
|
@ -523,3 +571,6 @@ Changelog
|
|||
[v16.1.3]: https://github.com/tobymao/sqlglot/compare/v16.1.2...v16.1.3
|
||||
[v16.1.4]: https://github.com/tobymao/sqlglot/compare/v16.1.3...v16.1.4
|
||||
[v16.2.0]: https://github.com/tobymao/sqlglot/compare/v16.1.4...v16.2.0
|
||||
[v16.2.1]: https://github.com/tobymao/sqlglot/compare/v16.2.0...v16.2.1
|
||||
[v16.3.0]: https://github.com/tobymao/sqlglot/compare/v16.2.1...v16.3.0
|
||||
[v16.3.1]: https://github.com/tobymao/sqlglot/compare/v16.3.0...v16.3.1
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -51,8 +51,8 @@
|
|||
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos">1</span></a><span class="c1"># file generated by setuptools_scm</span>
|
||||
</span><span id="L-2"><a href="#L-2"><span class="linenos">2</span></a><span class="c1"># don't change, don't track in version control</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos">3</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">'16.2.0'</span>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos">4</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">16</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos">3</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span> <span class="o">=</span> <span class="s1">'16.3.1'</span>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos">4</span></a><span class="n">__version_tuple__</span> <span class="o">=</span> <span class="n">version_tuple</span> <span class="o">=</span> <span class="p">(</span><span class="mi">16</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -72,7 +72,7 @@
|
|||
<ul>
|
||||
<li><a href="https://github.com/tobymao/sqlglot/blob/main/README.md#install">Install SQLGlot</a> and that is all that is required to just generate SQL. <a href="#examples">The examples</a> show generating SQL and then executing that SQL on a specific engine and that will require that engine's client library.</li>
|
||||
<li>Find/replace all <code>from pyspark.sql</code> with <code>from <a href="">sqlglot.dataframe</a></code>.</li>
|
||||
<li>Prior to any <code>spark.read.table</code> or <code>spark.table</code> run <code>sqlglot.schema.add_table('<table_name>', <column_structure>)</code>.
|
||||
<li>Prior to any <code>spark.read.table</code> or <code>spark.table</code> run <code>sqlglot.schema.add_table('<table_name>', <column_structure>, dialect="spark")</code>.
|
||||
<ul>
|
||||
<li>The column structure can be defined the following ways:
|
||||
<ul>
|
||||
|
|
@ -111,12 +111,16 @@
|
|||
<span class="kn">from</span> <span class="nn">sqlglot.dataframe.sql.session</span> <span class="kn">import</span> <span class="n">SparkSession</span>
|
||||
<span class="kn">from</span> <span class="nn"><a href="dataframe/sql.html">sqlglot.dataframe.sql</a></span> <span class="kn">import</span> <span class="n">functions</span> <span class="k">as</span> <span class="n">F</span>
|
||||
|
||||
<span class="n">sqlglot</span><span class="o">.</span><span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span><span class="s1">'employee'</span><span class="p">,</span> <span class="p">{</span>
|
||||
<span class="n">sqlglot</span><span class="o">.</span><span class="n">schema</span><span class="o">.</span><span class="n">add_table</span><span class="p">(</span>
|
||||
<span class="s1">'employee'</span><span class="p">,</span>
|
||||
<span class="p">{</span>
|
||||
<span class="s1">'employee_id'</span><span class="p">:</span> <span class="s1">'INT'</span><span class="p">,</span>
|
||||
<span class="s1">'fname'</span><span class="p">:</span> <span class="s1">'STRING'</span><span class="p">,</span>
|
||||
<span class="s1">'lname'</span><span class="p">:</span> <span class="s1">'STRING'</span><span class="p">,</span>
|
||||
<span class="s1">'age'</span><span class="p">:</span> <span class="s1">'INT'</span><span class="p">,</span>
|
||||
<span class="p">})</span> <span class="c1"># Register the table structure prior to reading from the table</span>
|
||||
<span class="p">},</span>
|
||||
<span class="n">dialect</span><span class="o">=</span><span class="s2">"spark"</span><span class="p">,</span>
|
||||
<span class="p">)</span> <span class="c1"># Register the table structure prior to reading from the table</span>
|
||||
|
||||
<span class="n">spark</span> <span class="o">=</span> <span class="n">SparkSession</span><span class="p">()</span>
|
||||
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -69,6 +69,9 @@
|
|||
<li>
|
||||
<a class="function" href="#ClickHouse.Generator.createable_sql">createable_sql</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="function" href="#ClickHouse.Generator.can_identify">can_identify</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</li>
|
||||
|
|
@ -445,7 +448,7 @@
|
|||
</span><span id="L-345"><a href="#L-345"><span class="linenos">345</span></a> <span class="s2">"CONCAT"</span><span class="p">,</span>
|
||||
</span><span id="L-346"><a href="#L-346"><span class="linenos">346</span></a> <span class="o">*</span><span class="p">[</span>
|
||||
</span><span id="L-347"><a href="#L-347"><span class="linenos">347</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">func</span><span class="p">(</span><span class="s2">"if"</span><span class="p">,</span> <span class="n">e</span><span class="o">.</span><span class="n">is_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">null</span><span class="p">()),</span> <span class="n">e</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="s2">"text"</span><span class="p">))</span>
|
||||
</span><span id="L-348"><a href="#L-348"><span class="linenos">348</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">expressions</span>
|
||||
</span><span id="L-348"><a href="#L-348"><span class="linenos">348</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Condition</span><span class="p">],</span> <span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span>
|
||||
</span><span id="L-349"><a href="#L-349"><span class="linenos">349</span></a> <span class="p">],</span>
|
||||
</span><span id="L-350"><a href="#L-350"><span class="linenos">350</span></a> <span class="p">)</span>
|
||||
</span><span id="L-351"><a href="#L-351"><span class="linenos">351</span></a>
|
||||
|
|
@ -829,7 +832,7 @@
|
|||
</span><span id="ClickHouse-346"><a href="#ClickHouse-346"><span class="linenos">346</span></a> <span class="s2">"CONCAT"</span><span class="p">,</span>
|
||||
</span><span id="ClickHouse-347"><a href="#ClickHouse-347"><span class="linenos">347</span></a> <span class="o">*</span><span class="p">[</span>
|
||||
</span><span id="ClickHouse-348"><a href="#ClickHouse-348"><span class="linenos">348</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">func</span><span class="p">(</span><span class="s2">"if"</span><span class="p">,</span> <span class="n">e</span><span class="o">.</span><span class="n">is_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">null</span><span class="p">()),</span> <span class="n">e</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="s2">"text"</span><span class="p">))</span>
|
||||
</span><span id="ClickHouse-349"><a href="#ClickHouse-349"><span class="linenos">349</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">expressions</span>
|
||||
</span><span id="ClickHouse-349"><a href="#ClickHouse-349"><span class="linenos">349</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Condition</span><span class="p">],</span> <span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span>
|
||||
</span><span id="ClickHouse-350"><a href="#ClickHouse-350"><span class="linenos">350</span></a> <span class="p">],</span>
|
||||
</span><span id="ClickHouse-351"><a href="#ClickHouse-351"><span class="linenos">351</span></a> <span class="p">)</span>
|
||||
</span><span id="ClickHouse-352"><a href="#ClickHouse-352"><span class="linenos">352</span></a>
|
||||
|
|
@ -884,6 +887,10 @@
|
|||
<div><dt><a href="dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></dt>
|
||||
<dd id="ClickHouse.get_or_raise" class="function"><a href="dialect.html#Dialect.get_or_raise">get_or_raise</a></dd>
|
||||
<dd id="ClickHouse.format_time" class="function"><a href="dialect.html#Dialect.format_time">format_time</a></dd>
|
||||
<dd id="ClickHouse.normalize_identifier" class="function"><a href="dialect.html#Dialect.normalize_identifier">normalize_identifier</a></dd>
|
||||
<dd id="ClickHouse.case_sensitive" class="function"><a href="dialect.html#Dialect.case_sensitive">case_sensitive</a></dd>
|
||||
<dd id="ClickHouse.can_identify" class="function"><a href="dialect.html#Dialect.can_identify">can_identify</a></dd>
|
||||
<dd id="ClickHouse.quote_identifier" class="function"><a href="dialect.html#Dialect.quote_identifier">quote_identifier</a></dd>
|
||||
<dd id="ClickHouse.parse" class="function"><a href="dialect.html#Dialect.parse">parse</a></dd>
|
||||
<dd id="ClickHouse.parse_into" class="function"><a href="dialect.html#Dialect.parse_into">parse_into</a></dd>
|
||||
<dd id="ClickHouse.generate" class="function"><a href="dialect.html#Dialect.generate">generate</a></dd>
|
||||
|
|
@ -1301,7 +1308,7 @@ Default: 3</li>
|
|||
</span><span id="ClickHouse.Generator-346"><a href="#ClickHouse.Generator-346"><span class="linenos">346</span></a> <span class="s2">"CONCAT"</span><span class="p">,</span>
|
||||
</span><span id="ClickHouse.Generator-347"><a href="#ClickHouse.Generator-347"><span class="linenos">347</span></a> <span class="o">*</span><span class="p">[</span>
|
||||
</span><span id="ClickHouse.Generator-348"><a href="#ClickHouse.Generator-348"><span class="linenos">348</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">func</span><span class="p">(</span><span class="s2">"if"</span><span class="p">,</span> <span class="n">e</span><span class="o">.</span><span class="n">is_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">null</span><span class="p">()),</span> <span class="n">e</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="s2">"text"</span><span class="p">))</span>
|
||||
</span><span id="ClickHouse.Generator-349"><a href="#ClickHouse.Generator-349"><span class="linenos">349</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">expressions</span>
|
||||
</span><span id="ClickHouse.Generator-349"><a href="#ClickHouse.Generator-349"><span class="linenos">349</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Condition</span><span class="p">],</span> <span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span>
|
||||
</span><span id="ClickHouse.Generator-350"><a href="#ClickHouse.Generator-350"><span class="linenos">350</span></a> <span class="p">],</span>
|
||||
</span><span id="ClickHouse.Generator-351"><a href="#ClickHouse.Generator-351"><span class="linenos">351</span></a> <span class="p">)</span>
|
||||
</span><span id="ClickHouse.Generator-352"><a href="#ClickHouse.Generator-352"><span class="linenos">352</span></a>
|
||||
|
|
@ -1404,7 +1411,7 @@ Default: True</li>
|
|||
</span><span id="ClickHouse.Generator.safeconcat_sql-346"><a href="#ClickHouse.Generator.safeconcat_sql-346"><span class="linenos">346</span></a> <span class="s2">"CONCAT"</span><span class="p">,</span>
|
||||
</span><span id="ClickHouse.Generator.safeconcat_sql-347"><a href="#ClickHouse.Generator.safeconcat_sql-347"><span class="linenos">347</span></a> <span class="o">*</span><span class="p">[</span>
|
||||
</span><span id="ClickHouse.Generator.safeconcat_sql-348"><a href="#ClickHouse.Generator.safeconcat_sql-348"><span class="linenos">348</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">func</span><span class="p">(</span><span class="s2">"if"</span><span class="p">,</span> <span class="n">e</span><span class="o">.</span><span class="n">is_</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">null</span><span class="p">()),</span> <span class="n">e</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">e</span><span class="p">,</span> <span class="s2">"text"</span><span class="p">))</span>
|
||||
</span><span id="ClickHouse.Generator.safeconcat_sql-349"><a href="#ClickHouse.Generator.safeconcat_sql-349"><span class="linenos">349</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">expressions</span>
|
||||
</span><span id="ClickHouse.Generator.safeconcat_sql-349"><a href="#ClickHouse.Generator.safeconcat_sql-349"><span class="linenos">349</span></a> <span class="k">for</span> <span class="n">e</span> <span class="ow">in</span> <span class="n">t</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Condition</span><span class="p">],</span> <span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span>
|
||||
</span><span id="ClickHouse.Generator.safeconcat_sql-350"><a href="#ClickHouse.Generator.safeconcat_sql-350"><span class="linenos">350</span></a> <span class="p">],</span>
|
||||
</span><span id="ClickHouse.Generator.safeconcat_sql-351"><a href="#ClickHouse.Generator.safeconcat_sql-351"><span class="linenos">351</span></a> <span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
|
@ -1549,6 +1556,60 @@ Default: True</li>
|
|||
|
||||
|
||||
|
||||
</div>
|
||||
<div id="ClickHouse.Generator.can_identify" class="classattr">
|
||||
<input id="ClickHouse.Generator.can_identify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">can_identify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">text</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s1">'safe'</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="ClickHouse.Generator.can_identify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#ClickHouse.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="ClickHouse.Generator.can_identify-247"><a href="#ClickHouse.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-248"><a href="#ClickHouse.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-249"><a href="#ClickHouse.Generator.can_identify-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-250"><a href="#ClickHouse.Generator.can_identify-250"><span class="linenos">250</span></a>
|
||||
</span><span id="ClickHouse.Generator.can_identify-251"><a href="#ClickHouse.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-252"><a href="#ClickHouse.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-253"><a href="#ClickHouse.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-254"><a href="#ClickHouse.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-255"><a href="#ClickHouse.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-256"><a href="#ClickHouse.Generator.can_identify-256"><span class="linenos">256</span></a>
|
||||
</span><span id="ClickHouse.Generator.can_identify-257"><a href="#ClickHouse.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-258"><a href="#ClickHouse.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-259"><a href="#ClickHouse.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> """</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-260"><a href="#ClickHouse.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-261"><a href="#ClickHouse.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-262"><a href="#ClickHouse.Generator.can_identify-262"><span class="linenos">262</span></a>
|
||||
</span><span id="ClickHouse.Generator.can_identify-263"><a href="#ClickHouse.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-264"><a href="#ClickHouse.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="ClickHouse.Generator.can_identify-265"><a href="#ClickHouse.Generator.can_identify-265"><span class="linenos">265</span></a>
|
||||
</span><span id="ClickHouse.Generator.can_identify-266"><a href="#ClickHouse.Generator.can_identify-266"><span class="linenos">266</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Checks if text can be identified given an identify option.</p>
|
||||
|
||||
<h6 id="arguments">Arguments:</h6>
|
||||
|
||||
<ul>
|
||||
<li><strong>text:</strong> The text to check.</li>
|
||||
<li><strong>identify:</strong> "always" or <code>True</code>: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Whether or not the given text can be identified.</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
<div class="inherited">
|
||||
<h5>Inherited Members</h5>
|
||||
|
|
@ -1650,6 +1711,7 @@ Default: True</li>
|
|||
<dd id="ClickHouse.Generator.pragma_sql" class="function"><a href="../generator.html#Generator.pragma_sql">pragma_sql</a></dd>
|
||||
<dd id="ClickHouse.Generator.lock_sql" class="function"><a href="../generator.html#Generator.lock_sql">lock_sql</a></dd>
|
||||
<dd id="ClickHouse.Generator.literal_sql" class="function"><a href="../generator.html#Generator.literal_sql">literal_sql</a></dd>
|
||||
<dd id="ClickHouse.Generator.escape_str" class="function"><a href="../generator.html#Generator.escape_str">escape_str</a></dd>
|
||||
<dd id="ClickHouse.Generator.loaddata_sql" class="function"><a href="../generator.html#Generator.loaddata_sql">loaddata_sql</a></dd>
|
||||
<dd id="ClickHouse.Generator.null_sql" class="function"><a href="../generator.html#Generator.null_sql">null_sql</a></dd>
|
||||
<dd id="ClickHouse.Generator.boolean_sql" class="function"><a href="../generator.html#Generator.boolean_sql">boolean_sql</a></dd>
|
||||
|
|
|
|||
|
|
@ -42,6 +42,9 @@
|
|||
<li>
|
||||
<a class="class" href="#Databricks.Generator">Databricks.Generator</a>
|
||||
<ul class="memberlist">
|
||||
<li>
|
||||
<a class="function" href="#Databricks.Generator.can_identify">can_identify</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</li>
|
||||
|
|
@ -192,6 +195,10 @@
|
|||
<div><dt><a href="dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></dt>
|
||||
<dd id="Databricks.get_or_raise" class="function"><a href="dialect.html#Dialect.get_or_raise">get_or_raise</a></dd>
|
||||
<dd id="Databricks.format_time" class="function"><a href="dialect.html#Dialect.format_time">format_time</a></dd>
|
||||
<dd id="Databricks.normalize_identifier" class="function"><a href="dialect.html#Dialect.normalize_identifier">normalize_identifier</a></dd>
|
||||
<dd id="Databricks.case_sensitive" class="function"><a href="dialect.html#Dialect.case_sensitive">case_sensitive</a></dd>
|
||||
<dd id="Databricks.can_identify" class="function"><a href="dialect.html#Dialect.can_identify">can_identify</a></dd>
|
||||
<dd id="Databricks.quote_identifier" class="function"><a href="dialect.html#Dialect.quote_identifier">quote_identifier</a></dd>
|
||||
<dd id="Databricks.parse" class="function"><a href="dialect.html#Dialect.parse">parse</a></dd>
|
||||
<dd id="Databricks.parse_into" class="function"><a href="dialect.html#Dialect.parse_into">parse_into</a></dd>
|
||||
<dd id="Databricks.generate" class="function"><a href="dialect.html#Dialect.generate">generate</a></dd>
|
||||
|
|
@ -335,6 +342,60 @@ Default: True</li>
|
|||
</div>
|
||||
|
||||
|
||||
<div id="Databricks.Generator.can_identify" class="classattr">
|
||||
<input id="Databricks.Generator.can_identify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">can_identify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">text</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s1">'safe'</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Databricks.Generator.can_identify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#Databricks.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Databricks.Generator.can_identify-247"><a href="#Databricks.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="Databricks.Generator.can_identify-248"><a href="#Databricks.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="Databricks.Generator.can_identify-249"><a href="#Databricks.Generator.can_identify-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="Databricks.Generator.can_identify-250"><a href="#Databricks.Generator.can_identify-250"><span class="linenos">250</span></a>
|
||||
</span><span id="Databricks.Generator.can_identify-251"><a href="#Databricks.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Databricks.Generator.can_identify-252"><a href="#Databricks.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="Databricks.Generator.can_identify-253"><a href="#Databricks.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="Databricks.Generator.can_identify-254"><a href="#Databricks.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="Databricks.Generator.can_identify-255"><a href="#Databricks.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="Databricks.Generator.can_identify-256"><a href="#Databricks.Generator.can_identify-256"><span class="linenos">256</span></a>
|
||||
</span><span id="Databricks.Generator.can_identify-257"><a href="#Databricks.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Databricks.Generator.can_identify-258"><a href="#Databricks.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="Databricks.Generator.can_identify-259"><a href="#Databricks.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> """</span>
|
||||
</span><span id="Databricks.Generator.can_identify-260"><a href="#Databricks.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="Databricks.Generator.can_identify-261"><a href="#Databricks.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="Databricks.Generator.can_identify-262"><a href="#Databricks.Generator.can_identify-262"><span class="linenos">262</span></a>
|
||||
</span><span id="Databricks.Generator.can_identify-263"><a href="#Databricks.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="Databricks.Generator.can_identify-264"><a href="#Databricks.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="Databricks.Generator.can_identify-265"><a href="#Databricks.Generator.can_identify-265"><span class="linenos">265</span></a>
|
||||
</span><span id="Databricks.Generator.can_identify-266"><a href="#Databricks.Generator.can_identify-266"><span class="linenos">266</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Checks if text can be identified given an identify option.</p>
|
||||
|
||||
<h6 id="arguments">Arguments:</h6>
|
||||
|
||||
<ul>
|
||||
<li><strong>text:</strong> The text to check.</li>
|
||||
<li><strong>identify:</strong> "always" or <code>True</code>: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Whether or not the given text can be identified.</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
<div class="inherited">
|
||||
<h5>Inherited Members</h5>
|
||||
<dl>
|
||||
|
|
@ -434,6 +495,7 @@ Default: True</li>
|
|||
<dd id="Databricks.Generator.pragma_sql" class="function"><a href="../generator.html#Generator.pragma_sql">pragma_sql</a></dd>
|
||||
<dd id="Databricks.Generator.lock_sql" class="function"><a href="../generator.html#Generator.lock_sql">lock_sql</a></dd>
|
||||
<dd id="Databricks.Generator.literal_sql" class="function"><a href="../generator.html#Generator.literal_sql">literal_sql</a></dd>
|
||||
<dd id="Databricks.Generator.escape_str" class="function"><a href="../generator.html#Generator.escape_str">escape_str</a></dd>
|
||||
<dd id="Databricks.Generator.loaddata_sql" class="function"><a href="../generator.html#Generator.loaddata_sql">loaddata_sql</a></dd>
|
||||
<dd id="Databricks.Generator.null_sql" class="function"><a href="../generator.html#Generator.null_sql">null_sql</a></dd>
|
||||
<dd id="Databricks.Generator.boolean_sql" class="function"><a href="../generator.html#Generator.boolean_sql">boolean_sql</a></dd>
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -51,6 +51,9 @@
|
|||
<li>
|
||||
<a class="function" href="#Drill.Generator.normalize_func">normalize_func</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="function" href="#Drill.Generator.can_identify">can_identify</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</li>
|
||||
|
|
@ -375,6 +378,10 @@
|
|||
<div><dt><a href="dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></dt>
|
||||
<dd id="Drill.get_or_raise" class="function"><a href="dialect.html#Dialect.get_or_raise">get_or_raise</a></dd>
|
||||
<dd id="Drill.format_time" class="function"><a href="dialect.html#Dialect.format_time">format_time</a></dd>
|
||||
<dd id="Drill.normalize_identifier" class="function"><a href="dialect.html#Dialect.normalize_identifier">normalize_identifier</a></dd>
|
||||
<dd id="Drill.case_sensitive" class="function"><a href="dialect.html#Dialect.case_sensitive">case_sensitive</a></dd>
|
||||
<dd id="Drill.can_identify" class="function"><a href="dialect.html#Dialect.can_identify">can_identify</a></dd>
|
||||
<dd id="Drill.quote_identifier" class="function"><a href="dialect.html#Dialect.quote_identifier">quote_identifier</a></dd>
|
||||
<dd id="Drill.parse" class="function"><a href="dialect.html#Dialect.parse">parse</a></dd>
|
||||
<dd id="Drill.parse_into" class="function"><a href="dialect.html#Dialect.parse_into">parse_into</a></dd>
|
||||
<dd id="Drill.generate" class="function"><a href="dialect.html#Dialect.generate">generate</a></dd>
|
||||
|
|
@ -609,6 +616,60 @@ Default: True</li>
|
|||
|
||||
|
||||
|
||||
</div>
|
||||
<div id="Drill.Generator.can_identify" class="classattr">
|
||||
<input id="Drill.Generator.can_identify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">can_identify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">text</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s1">'safe'</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Drill.Generator.can_identify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#Drill.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Drill.Generator.can_identify-247"><a href="#Drill.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="Drill.Generator.can_identify-248"><a href="#Drill.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="Drill.Generator.can_identify-249"><a href="#Drill.Generator.can_identify-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="Drill.Generator.can_identify-250"><a href="#Drill.Generator.can_identify-250"><span class="linenos">250</span></a>
|
||||
</span><span id="Drill.Generator.can_identify-251"><a href="#Drill.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Drill.Generator.can_identify-252"><a href="#Drill.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="Drill.Generator.can_identify-253"><a href="#Drill.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="Drill.Generator.can_identify-254"><a href="#Drill.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="Drill.Generator.can_identify-255"><a href="#Drill.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="Drill.Generator.can_identify-256"><a href="#Drill.Generator.can_identify-256"><span class="linenos">256</span></a>
|
||||
</span><span id="Drill.Generator.can_identify-257"><a href="#Drill.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Drill.Generator.can_identify-258"><a href="#Drill.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="Drill.Generator.can_identify-259"><a href="#Drill.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> """</span>
|
||||
</span><span id="Drill.Generator.can_identify-260"><a href="#Drill.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="Drill.Generator.can_identify-261"><a href="#Drill.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="Drill.Generator.can_identify-262"><a href="#Drill.Generator.can_identify-262"><span class="linenos">262</span></a>
|
||||
</span><span id="Drill.Generator.can_identify-263"><a href="#Drill.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="Drill.Generator.can_identify-264"><a href="#Drill.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="Drill.Generator.can_identify-265"><a href="#Drill.Generator.can_identify-265"><span class="linenos">265</span></a>
|
||||
</span><span id="Drill.Generator.can_identify-266"><a href="#Drill.Generator.can_identify-266"><span class="linenos">266</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Checks if text can be identified given an identify option.</p>
|
||||
|
||||
<h6 id="arguments">Arguments:</h6>
|
||||
|
||||
<ul>
|
||||
<li><strong>text:</strong> The text to check.</li>
|
||||
<li><strong>identify:</strong> "always" or <code>True</code>: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Whether or not the given text can be identified.</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
<div class="inherited">
|
||||
<h5>Inherited Members</h5>
|
||||
|
|
@ -711,6 +772,7 @@ Default: True</li>
|
|||
<dd id="Drill.Generator.pragma_sql" class="function"><a href="../generator.html#Generator.pragma_sql">pragma_sql</a></dd>
|
||||
<dd id="Drill.Generator.lock_sql" class="function"><a href="../generator.html#Generator.lock_sql">lock_sql</a></dd>
|
||||
<dd id="Drill.Generator.literal_sql" class="function"><a href="../generator.html#Generator.literal_sql">literal_sql</a></dd>
|
||||
<dd id="Drill.Generator.escape_str" class="function"><a href="../generator.html#Generator.escape_str">escape_str</a></dd>
|
||||
<dd id="Drill.Generator.loaddata_sql" class="function"><a href="../generator.html#Generator.loaddata_sql">loaddata_sql</a></dd>
|
||||
<dd id="Drill.Generator.null_sql" class="function"><a href="../generator.html#Generator.null_sql">null_sql</a></dd>
|
||||
<dd id="Drill.Generator.boolean_sql" class="function"><a href="../generator.html#Generator.boolean_sql">boolean_sql</a></dd>
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -51,6 +51,9 @@
|
|||
<li>
|
||||
<a class="function" href="#MySQL.Generator.show_sql">show_sql</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="function" href="#MySQL.Generator.can_identify">can_identify</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</li>
|
||||
|
|
@ -989,6 +992,10 @@
|
|||
<div><dt><a href="dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></dt>
|
||||
<dd id="MySQL.get_or_raise" class="function"><a href="dialect.html#Dialect.get_or_raise">get_or_raise</a></dd>
|
||||
<dd id="MySQL.format_time" class="function"><a href="dialect.html#Dialect.format_time">format_time</a></dd>
|
||||
<dd id="MySQL.normalize_identifier" class="function"><a href="dialect.html#Dialect.normalize_identifier">normalize_identifier</a></dd>
|
||||
<dd id="MySQL.case_sensitive" class="function"><a href="dialect.html#Dialect.case_sensitive">case_sensitive</a></dd>
|
||||
<dd id="MySQL.can_identify" class="function"><a href="dialect.html#Dialect.can_identify">can_identify</a></dd>
|
||||
<dd id="MySQL.quote_identifier" class="function"><a href="dialect.html#Dialect.quote_identifier">quote_identifier</a></dd>
|
||||
<dd id="MySQL.parse" class="function"><a href="dialect.html#Dialect.parse">parse</a></dd>
|
||||
<dd id="MySQL.parse_into" class="function"><a href="dialect.html#Dialect.parse_into">parse_into</a></dd>
|
||||
<dd id="MySQL.generate" class="function"><a href="dialect.html#Dialect.generate">generate</a></dd>
|
||||
|
|
@ -1558,6 +1565,60 @@ Default: True</li>
|
|||
|
||||
|
||||
|
||||
</div>
|
||||
<div id="MySQL.Generator.can_identify" class="classattr">
|
||||
<input id="MySQL.Generator.can_identify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">can_identify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">text</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s1">'safe'</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="MySQL.Generator.can_identify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#MySQL.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="MySQL.Generator.can_identify-247"><a href="#MySQL.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="MySQL.Generator.can_identify-248"><a href="#MySQL.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="MySQL.Generator.can_identify-249"><a href="#MySQL.Generator.can_identify-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="MySQL.Generator.can_identify-250"><a href="#MySQL.Generator.can_identify-250"><span class="linenos">250</span></a>
|
||||
</span><span id="MySQL.Generator.can_identify-251"><a href="#MySQL.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="MySQL.Generator.can_identify-252"><a href="#MySQL.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="MySQL.Generator.can_identify-253"><a href="#MySQL.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="MySQL.Generator.can_identify-254"><a href="#MySQL.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="MySQL.Generator.can_identify-255"><a href="#MySQL.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="MySQL.Generator.can_identify-256"><a href="#MySQL.Generator.can_identify-256"><span class="linenos">256</span></a>
|
||||
</span><span id="MySQL.Generator.can_identify-257"><a href="#MySQL.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="MySQL.Generator.can_identify-258"><a href="#MySQL.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="MySQL.Generator.can_identify-259"><a href="#MySQL.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> """</span>
|
||||
</span><span id="MySQL.Generator.can_identify-260"><a href="#MySQL.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="MySQL.Generator.can_identify-261"><a href="#MySQL.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="MySQL.Generator.can_identify-262"><a href="#MySQL.Generator.can_identify-262"><span class="linenos">262</span></a>
|
||||
</span><span id="MySQL.Generator.can_identify-263"><a href="#MySQL.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="MySQL.Generator.can_identify-264"><a href="#MySQL.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="MySQL.Generator.can_identify-265"><a href="#MySQL.Generator.can_identify-265"><span class="linenos">265</span></a>
|
||||
</span><span id="MySQL.Generator.can_identify-266"><a href="#MySQL.Generator.can_identify-266"><span class="linenos">266</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Checks if text can be identified given an identify option.</p>
|
||||
|
||||
<h6 id="arguments">Arguments:</h6>
|
||||
|
||||
<ul>
|
||||
<li><strong>text:</strong> The text to check.</li>
|
||||
<li><strong>identify:</strong> "always" or <code>True</code>: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Whether or not the given text can be identified.</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
<div class="inherited">
|
||||
<h5>Inherited Members</h5>
|
||||
|
|
@ -1661,6 +1722,7 @@ Default: True</li>
|
|||
<dd id="MySQL.Generator.pragma_sql" class="function"><a href="../generator.html#Generator.pragma_sql">pragma_sql</a></dd>
|
||||
<dd id="MySQL.Generator.lock_sql" class="function"><a href="../generator.html#Generator.lock_sql">lock_sql</a></dd>
|
||||
<dd id="MySQL.Generator.literal_sql" class="function"><a href="../generator.html#Generator.literal_sql">literal_sql</a></dd>
|
||||
<dd id="MySQL.Generator.escape_str" class="function"><a href="../generator.html#Generator.escape_str">escape_str</a></dd>
|
||||
<dd id="MySQL.Generator.loaddata_sql" class="function"><a href="../generator.html#Generator.loaddata_sql">loaddata_sql</a></dd>
|
||||
<dd id="MySQL.Generator.null_sql" class="function"><a href="../generator.html#Generator.null_sql">null_sql</a></dd>
|
||||
<dd id="MySQL.Generator.boolean_sql" class="function"><a href="../generator.html#Generator.boolean_sql">boolean_sql</a></dd>
|
||||
|
|
|
|||
|
|
@ -51,6 +51,9 @@
|
|||
<li>
|
||||
<a class="function" href="#Oracle.Generator.xmltable_sql">xmltable_sql</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="function" href="#Oracle.Generator.can_identify">can_identify</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</li>
|
||||
|
|
@ -443,6 +446,10 @@
|
|||
<div><dt><a href="dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></dt>
|
||||
<dd id="Oracle.get_or_raise" class="function"><a href="dialect.html#Dialect.get_or_raise">get_or_raise</a></dd>
|
||||
<dd id="Oracle.format_time" class="function"><a href="dialect.html#Dialect.format_time">format_time</a></dd>
|
||||
<dd id="Oracle.normalize_identifier" class="function"><a href="dialect.html#Dialect.normalize_identifier">normalize_identifier</a></dd>
|
||||
<dd id="Oracle.case_sensitive" class="function"><a href="dialect.html#Dialect.case_sensitive">case_sensitive</a></dd>
|
||||
<dd id="Oracle.can_identify" class="function"><a href="dialect.html#Dialect.can_identify">can_identify</a></dd>
|
||||
<dd id="Oracle.quote_identifier" class="function"><a href="dialect.html#Dialect.quote_identifier">quote_identifier</a></dd>
|
||||
<dd id="Oracle.parse" class="function"><a href="dialect.html#Dialect.parse">parse</a></dd>
|
||||
<dd id="Oracle.parse_into" class="function"><a href="dialect.html#Dialect.parse_into">parse_into</a></dd>
|
||||
<dd id="Oracle.generate" class="function"><a href="dialect.html#Dialect.generate">generate</a></dd>
|
||||
|
|
@ -724,6 +731,60 @@ Default: True</li>
|
|||
|
||||
|
||||
|
||||
</div>
|
||||
<div id="Oracle.Generator.can_identify" class="classattr">
|
||||
<input id="Oracle.Generator.can_identify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">can_identify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">text</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s1">'safe'</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Oracle.Generator.can_identify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#Oracle.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Oracle.Generator.can_identify-247"><a href="#Oracle.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="Oracle.Generator.can_identify-248"><a href="#Oracle.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="Oracle.Generator.can_identify-249"><a href="#Oracle.Generator.can_identify-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="Oracle.Generator.can_identify-250"><a href="#Oracle.Generator.can_identify-250"><span class="linenos">250</span></a>
|
||||
</span><span id="Oracle.Generator.can_identify-251"><a href="#Oracle.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Oracle.Generator.can_identify-252"><a href="#Oracle.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="Oracle.Generator.can_identify-253"><a href="#Oracle.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="Oracle.Generator.can_identify-254"><a href="#Oracle.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="Oracle.Generator.can_identify-255"><a href="#Oracle.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="Oracle.Generator.can_identify-256"><a href="#Oracle.Generator.can_identify-256"><span class="linenos">256</span></a>
|
||||
</span><span id="Oracle.Generator.can_identify-257"><a href="#Oracle.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Oracle.Generator.can_identify-258"><a href="#Oracle.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="Oracle.Generator.can_identify-259"><a href="#Oracle.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> """</span>
|
||||
</span><span id="Oracle.Generator.can_identify-260"><a href="#Oracle.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="Oracle.Generator.can_identify-261"><a href="#Oracle.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="Oracle.Generator.can_identify-262"><a href="#Oracle.Generator.can_identify-262"><span class="linenos">262</span></a>
|
||||
</span><span id="Oracle.Generator.can_identify-263"><a href="#Oracle.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="Oracle.Generator.can_identify-264"><a href="#Oracle.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="Oracle.Generator.can_identify-265"><a href="#Oracle.Generator.can_identify-265"><span class="linenos">265</span></a>
|
||||
</span><span id="Oracle.Generator.can_identify-266"><a href="#Oracle.Generator.can_identify-266"><span class="linenos">266</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Checks if text can be identified given an identify option.</p>
|
||||
|
||||
<h6 id="arguments">Arguments:</h6>
|
||||
|
||||
<ul>
|
||||
<li><strong>text:</strong> The text to check.</li>
|
||||
<li><strong>identify:</strong> "always" or <code>True</code>: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Whether or not the given text can be identified.</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
<div class="inherited">
|
||||
<h5>Inherited Members</h5>
|
||||
|
|
@ -825,6 +886,7 @@ Default: True</li>
|
|||
<dd id="Oracle.Generator.pragma_sql" class="function"><a href="../generator.html#Generator.pragma_sql">pragma_sql</a></dd>
|
||||
<dd id="Oracle.Generator.lock_sql" class="function"><a href="../generator.html#Generator.lock_sql">lock_sql</a></dd>
|
||||
<dd id="Oracle.Generator.literal_sql" class="function"><a href="../generator.html#Generator.literal_sql">literal_sql</a></dd>
|
||||
<dd id="Oracle.Generator.escape_str" class="function"><a href="../generator.html#Generator.escape_str">escape_str</a></dd>
|
||||
<dd id="Oracle.Generator.loaddata_sql" class="function"><a href="../generator.html#Generator.loaddata_sql">loaddata_sql</a></dd>
|
||||
<dd id="Oracle.Generator.null_sql" class="function"><a href="../generator.html#Generator.null_sql">null_sql</a></dd>
|
||||
<dd id="Oracle.Generator.boolean_sql" class="function"><a href="../generator.html#Generator.boolean_sql">boolean_sql</a></dd>
|
||||
|
|
|
|||
|
|
@ -48,6 +48,9 @@
|
|||
<li>
|
||||
<a class="class" href="#Postgres.Generator">Postgres.Generator</a>
|
||||
<ul class="memberlist">
|
||||
<li>
|
||||
<a class="function" href="#Postgres.Generator.can_identify">can_identify</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</li>
|
||||
|
|
@ -679,6 +682,10 @@
|
|||
<div><dt><a href="dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></dt>
|
||||
<dd id="Postgres.get_or_raise" class="function"><a href="dialect.html#Dialect.get_or_raise">get_or_raise</a></dd>
|
||||
<dd id="Postgres.format_time" class="function"><a href="dialect.html#Dialect.format_time">format_time</a></dd>
|
||||
<dd id="Postgres.normalize_identifier" class="function"><a href="dialect.html#Dialect.normalize_identifier">normalize_identifier</a></dd>
|
||||
<dd id="Postgres.case_sensitive" class="function"><a href="dialect.html#Dialect.case_sensitive">case_sensitive</a></dd>
|
||||
<dd id="Postgres.can_identify" class="function"><a href="dialect.html#Dialect.can_identify">can_identify</a></dd>
|
||||
<dd id="Postgres.quote_identifier" class="function"><a href="dialect.html#Dialect.quote_identifier">quote_identifier</a></dd>
|
||||
<dd id="Postgres.parse" class="function"><a href="dialect.html#Dialect.parse">parse</a></dd>
|
||||
<dd id="Postgres.parse_into" class="function"><a href="dialect.html#Dialect.parse_into">parse_into</a></dd>
|
||||
<dd id="Postgres.generate" class="function"><a href="dialect.html#Dialect.generate">generate</a></dd>
|
||||
|
|
@ -980,6 +987,60 @@ Default: True</li>
|
|||
</div>
|
||||
|
||||
|
||||
<div id="Postgres.Generator.can_identify" class="classattr">
|
||||
<input id="Postgres.Generator.can_identify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">can_identify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">text</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s1">'safe'</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Postgres.Generator.can_identify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#Postgres.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Postgres.Generator.can_identify-247"><a href="#Postgres.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="Postgres.Generator.can_identify-248"><a href="#Postgres.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="Postgres.Generator.can_identify-249"><a href="#Postgres.Generator.can_identify-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="Postgres.Generator.can_identify-250"><a href="#Postgres.Generator.can_identify-250"><span class="linenos">250</span></a>
|
||||
</span><span id="Postgres.Generator.can_identify-251"><a href="#Postgres.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Postgres.Generator.can_identify-252"><a href="#Postgres.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="Postgres.Generator.can_identify-253"><a href="#Postgres.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="Postgres.Generator.can_identify-254"><a href="#Postgres.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="Postgres.Generator.can_identify-255"><a href="#Postgres.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="Postgres.Generator.can_identify-256"><a href="#Postgres.Generator.can_identify-256"><span class="linenos">256</span></a>
|
||||
</span><span id="Postgres.Generator.can_identify-257"><a href="#Postgres.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Postgres.Generator.can_identify-258"><a href="#Postgres.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="Postgres.Generator.can_identify-259"><a href="#Postgres.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> """</span>
|
||||
</span><span id="Postgres.Generator.can_identify-260"><a href="#Postgres.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="Postgres.Generator.can_identify-261"><a href="#Postgres.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="Postgres.Generator.can_identify-262"><a href="#Postgres.Generator.can_identify-262"><span class="linenos">262</span></a>
|
||||
</span><span id="Postgres.Generator.can_identify-263"><a href="#Postgres.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="Postgres.Generator.can_identify-264"><a href="#Postgres.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="Postgres.Generator.can_identify-265"><a href="#Postgres.Generator.can_identify-265"><span class="linenos">265</span></a>
|
||||
</span><span id="Postgres.Generator.can_identify-266"><a href="#Postgres.Generator.can_identify-266"><span class="linenos">266</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Checks if text can be identified given an identify option.</p>
|
||||
|
||||
<h6 id="arguments">Arguments:</h6>
|
||||
|
||||
<ul>
|
||||
<li><strong>text:</strong> The text to check.</li>
|
||||
<li><strong>identify:</strong> "always" or <code>True</code>: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Whether or not the given text can be identified.</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
<div class="inherited">
|
||||
<h5>Inherited Members</h5>
|
||||
<dl>
|
||||
|
|
@ -1082,6 +1143,7 @@ Default: True</li>
|
|||
<dd id="Postgres.Generator.pragma_sql" class="function"><a href="../generator.html#Generator.pragma_sql">pragma_sql</a></dd>
|
||||
<dd id="Postgres.Generator.lock_sql" class="function"><a href="../generator.html#Generator.lock_sql">lock_sql</a></dd>
|
||||
<dd id="Postgres.Generator.literal_sql" class="function"><a href="../generator.html#Generator.literal_sql">literal_sql</a></dd>
|
||||
<dd id="Postgres.Generator.escape_str" class="function"><a href="../generator.html#Generator.escape_str">escape_str</a></dd>
|
||||
<dd id="Postgres.Generator.loaddata_sql" class="function"><a href="../generator.html#Generator.loaddata_sql">loaddata_sql</a></dd>
|
||||
<dd id="Postgres.Generator.null_sql" class="function"><a href="../generator.html#Generator.null_sql">null_sql</a></dd>
|
||||
<dd id="Postgres.Generator.boolean_sql" class="function"><a href="../generator.html#Generator.boolean_sql">boolean_sql</a></dd>
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -45,6 +45,9 @@
|
|||
<li>
|
||||
<a class="function" href="#Spark.Generator.datediff_sql">datediff_sql</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="function" href="#Spark.Generator.can_identify">can_identify</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</li>
|
||||
|
|
@ -179,6 +182,10 @@
|
|||
<div><dt><a href="dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></dt>
|
||||
<dd id="Spark.get_or_raise" class="function"><a href="dialect.html#Dialect.get_or_raise">get_or_raise</a></dd>
|
||||
<dd id="Spark.format_time" class="function"><a href="dialect.html#Dialect.format_time">format_time</a></dd>
|
||||
<dd id="Spark.normalize_identifier" class="function"><a href="dialect.html#Dialect.normalize_identifier">normalize_identifier</a></dd>
|
||||
<dd id="Spark.case_sensitive" class="function"><a href="dialect.html#Dialect.case_sensitive">case_sensitive</a></dd>
|
||||
<dd id="Spark.can_identify" class="function"><a href="dialect.html#Dialect.can_identify">can_identify</a></dd>
|
||||
<dd id="Spark.quote_identifier" class="function"><a href="dialect.html#Dialect.quote_identifier">quote_identifier</a></dd>
|
||||
<dd id="Spark.parse" class="function"><a href="dialect.html#Dialect.parse">parse</a></dd>
|
||||
<dd id="Spark.parse_into" class="function"><a href="dialect.html#Dialect.parse_into">parse_into</a></dd>
|
||||
<dd id="Spark.generate" class="function"><a href="dialect.html#Dialect.generate">generate</a></dd>
|
||||
|
|
@ -335,6 +342,60 @@ Default: True</li>
|
|||
|
||||
|
||||
|
||||
</div>
|
||||
<div id="Spark.Generator.can_identify" class="classattr">
|
||||
<input id="Spark.Generator.can_identify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">can_identify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">text</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s1">'safe'</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Spark.Generator.can_identify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#Spark.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Spark.Generator.can_identify-247"><a href="#Spark.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="Spark.Generator.can_identify-248"><a href="#Spark.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="Spark.Generator.can_identify-249"><a href="#Spark.Generator.can_identify-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="Spark.Generator.can_identify-250"><a href="#Spark.Generator.can_identify-250"><span class="linenos">250</span></a>
|
||||
</span><span id="Spark.Generator.can_identify-251"><a href="#Spark.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Spark.Generator.can_identify-252"><a href="#Spark.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="Spark.Generator.can_identify-253"><a href="#Spark.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="Spark.Generator.can_identify-254"><a href="#Spark.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="Spark.Generator.can_identify-255"><a href="#Spark.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="Spark.Generator.can_identify-256"><a href="#Spark.Generator.can_identify-256"><span class="linenos">256</span></a>
|
||||
</span><span id="Spark.Generator.can_identify-257"><a href="#Spark.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Spark.Generator.can_identify-258"><a href="#Spark.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="Spark.Generator.can_identify-259"><a href="#Spark.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> """</span>
|
||||
</span><span id="Spark.Generator.can_identify-260"><a href="#Spark.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="Spark.Generator.can_identify-261"><a href="#Spark.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="Spark.Generator.can_identify-262"><a href="#Spark.Generator.can_identify-262"><span class="linenos">262</span></a>
|
||||
</span><span id="Spark.Generator.can_identify-263"><a href="#Spark.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="Spark.Generator.can_identify-264"><a href="#Spark.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="Spark.Generator.can_identify-265"><a href="#Spark.Generator.can_identify-265"><span class="linenos">265</span></a>
|
||||
</span><span id="Spark.Generator.can_identify-266"><a href="#Spark.Generator.can_identify-266"><span class="linenos">266</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Checks if text can be identified given an identify option.</p>
|
||||
|
||||
<h6 id="arguments">Arguments:</h6>
|
||||
|
||||
<ul>
|
||||
<li><strong>text:</strong> The text to check.</li>
|
||||
<li><strong>identify:</strong> "always" or <code>True</code>: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Whether or not the given text can be identified.</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
<div class="inherited">
|
||||
<h5>Inherited Members</h5>
|
||||
|
|
@ -435,6 +496,7 @@ Default: True</li>
|
|||
<dd id="Spark.Generator.pragma_sql" class="function"><a href="../generator.html#Generator.pragma_sql">pragma_sql</a></dd>
|
||||
<dd id="Spark.Generator.lock_sql" class="function"><a href="../generator.html#Generator.lock_sql">lock_sql</a></dd>
|
||||
<dd id="Spark.Generator.literal_sql" class="function"><a href="../generator.html#Generator.literal_sql">literal_sql</a></dd>
|
||||
<dd id="Spark.Generator.escape_str" class="function"><a href="../generator.html#Generator.escape_str">escape_str</a></dd>
|
||||
<dd id="Spark.Generator.loaddata_sql" class="function"><a href="../generator.html#Generator.loaddata_sql">loaddata_sql</a></dd>
|
||||
<dd id="Spark.Generator.null_sql" class="function"><a href="../generator.html#Generator.null_sql">null_sql</a></dd>
|
||||
<dd id="Spark.Generator.boolean_sql" class="function"><a href="../generator.html#Generator.boolean_sql">boolean_sql</a></dd>
|
||||
|
|
|
|||
|
|
@ -48,6 +48,9 @@
|
|||
<li>
|
||||
<a class="function" href="#Spark2.Generator.columndef_sql">columndef_sql</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="function" href="#Spark2.Generator.can_identify">can_identify</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</li>
|
||||
|
|
@ -509,6 +512,10 @@
|
|||
<div><dt><a href="dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></dt>
|
||||
<dd id="Spark2.get_or_raise" class="function"><a href="dialect.html#Dialect.get_or_raise">get_or_raise</a></dd>
|
||||
<dd id="Spark2.format_time" class="function"><a href="dialect.html#Dialect.format_time">format_time</a></dd>
|
||||
<dd id="Spark2.normalize_identifier" class="function"><a href="dialect.html#Dialect.normalize_identifier">normalize_identifier</a></dd>
|
||||
<dd id="Spark2.case_sensitive" class="function"><a href="dialect.html#Dialect.case_sensitive">case_sensitive</a></dd>
|
||||
<dd id="Spark2.can_identify" class="function"><a href="dialect.html#Dialect.can_identify">can_identify</a></dd>
|
||||
<dd id="Spark2.quote_identifier" class="function"><a href="dialect.html#Dialect.quote_identifier">quote_identifier</a></dd>
|
||||
<dd id="Spark2.parse" class="function"><a href="dialect.html#Dialect.parse">parse</a></dd>
|
||||
<dd id="Spark2.parse_into" class="function"><a href="dialect.html#Dialect.parse_into">parse_into</a></dd>
|
||||
<dd id="Spark2.generate" class="function"><a href="dialect.html#Dialect.generate">generate</a></dd>
|
||||
|
|
@ -813,6 +820,60 @@ Default: True</li>
|
|||
|
||||
|
||||
|
||||
</div>
|
||||
<div id="Spark2.Generator.can_identify" class="classattr">
|
||||
<input id="Spark2.Generator.can_identify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">can_identify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">text</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s1">'safe'</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Spark2.Generator.can_identify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#Spark2.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Spark2.Generator.can_identify-247"><a href="#Spark2.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="Spark2.Generator.can_identify-248"><a href="#Spark2.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="Spark2.Generator.can_identify-249"><a href="#Spark2.Generator.can_identify-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="Spark2.Generator.can_identify-250"><a href="#Spark2.Generator.can_identify-250"><span class="linenos">250</span></a>
|
||||
</span><span id="Spark2.Generator.can_identify-251"><a href="#Spark2.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Spark2.Generator.can_identify-252"><a href="#Spark2.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="Spark2.Generator.can_identify-253"><a href="#Spark2.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="Spark2.Generator.can_identify-254"><a href="#Spark2.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="Spark2.Generator.can_identify-255"><a href="#Spark2.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="Spark2.Generator.can_identify-256"><a href="#Spark2.Generator.can_identify-256"><span class="linenos">256</span></a>
|
||||
</span><span id="Spark2.Generator.can_identify-257"><a href="#Spark2.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Spark2.Generator.can_identify-258"><a href="#Spark2.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="Spark2.Generator.can_identify-259"><a href="#Spark2.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> """</span>
|
||||
</span><span id="Spark2.Generator.can_identify-260"><a href="#Spark2.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="Spark2.Generator.can_identify-261"><a href="#Spark2.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="Spark2.Generator.can_identify-262"><a href="#Spark2.Generator.can_identify-262"><span class="linenos">262</span></a>
|
||||
</span><span id="Spark2.Generator.can_identify-263"><a href="#Spark2.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="Spark2.Generator.can_identify-264"><a href="#Spark2.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="Spark2.Generator.can_identify-265"><a href="#Spark2.Generator.can_identify-265"><span class="linenos">265</span></a>
|
||||
</span><span id="Spark2.Generator.can_identify-266"><a href="#Spark2.Generator.can_identify-266"><span class="linenos">266</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Checks if text can be identified given an identify option.</p>
|
||||
|
||||
<h6 id="arguments">Arguments:</h6>
|
||||
|
||||
<ul>
|
||||
<li><strong>text:</strong> The text to check.</li>
|
||||
<li><strong>identify:</strong> "always" or <code>True</code>: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Whether or not the given text can be identified.</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
<div class="inherited">
|
||||
<h5>Inherited Members</h5>
|
||||
|
|
@ -913,6 +974,7 @@ Default: True</li>
|
|||
<dd id="Spark2.Generator.pragma_sql" class="function"><a href="../generator.html#Generator.pragma_sql">pragma_sql</a></dd>
|
||||
<dd id="Spark2.Generator.lock_sql" class="function"><a href="../generator.html#Generator.lock_sql">lock_sql</a></dd>
|
||||
<dd id="Spark2.Generator.literal_sql" class="function"><a href="../generator.html#Generator.literal_sql">literal_sql</a></dd>
|
||||
<dd id="Spark2.Generator.escape_str" class="function"><a href="../generator.html#Generator.escape_str">escape_str</a></dd>
|
||||
<dd id="Spark2.Generator.loaddata_sql" class="function"><a href="../generator.html#Generator.loaddata_sql">loaddata_sql</a></dd>
|
||||
<dd id="Spark2.Generator.null_sql" class="function"><a href="../generator.html#Generator.null_sql">null_sql</a></dd>
|
||||
<dd id="Spark2.Generator.boolean_sql" class="function"><a href="../generator.html#Generator.boolean_sql">boolean_sql</a></dd>
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -42,6 +42,9 @@
|
|||
<li>
|
||||
<a class="class" href="#StarRocks.Generator">StarRocks.Generator</a>
|
||||
<ul class="memberlist">
|
||||
<li>
|
||||
<a class="function" href="#StarRocks.Generator.can_identify">can_identify</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</li>
|
||||
|
|
@ -181,6 +184,10 @@
|
|||
<div><dt><a href="dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></dt>
|
||||
<dd id="StarRocks.get_or_raise" class="function"><a href="dialect.html#Dialect.get_or_raise">get_or_raise</a></dd>
|
||||
<dd id="StarRocks.format_time" class="function"><a href="dialect.html#Dialect.format_time">format_time</a></dd>
|
||||
<dd id="StarRocks.normalize_identifier" class="function"><a href="dialect.html#Dialect.normalize_identifier">normalize_identifier</a></dd>
|
||||
<dd id="StarRocks.case_sensitive" class="function"><a href="dialect.html#Dialect.case_sensitive">case_sensitive</a></dd>
|
||||
<dd id="StarRocks.can_identify" class="function"><a href="dialect.html#Dialect.can_identify">can_identify</a></dd>
|
||||
<dd id="StarRocks.quote_identifier" class="function"><a href="dialect.html#Dialect.quote_identifier">quote_identifier</a></dd>
|
||||
<dd id="StarRocks.parse" class="function"><a href="dialect.html#Dialect.parse">parse</a></dd>
|
||||
<dd id="StarRocks.parse_into" class="function"><a href="dialect.html#Dialect.parse_into">parse_into</a></dd>
|
||||
<dd id="StarRocks.generate" class="function"><a href="dialect.html#Dialect.generate">generate</a></dd>
|
||||
|
|
@ -326,6 +333,60 @@ Default: True</li>
|
|||
</div>
|
||||
|
||||
|
||||
<div id="StarRocks.Generator.can_identify" class="classattr">
|
||||
<input id="StarRocks.Generator.can_identify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">can_identify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">text</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s1">'safe'</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="StarRocks.Generator.can_identify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#StarRocks.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="StarRocks.Generator.can_identify-247"><a href="#StarRocks.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-248"><a href="#StarRocks.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-249"><a href="#StarRocks.Generator.can_identify-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-250"><a href="#StarRocks.Generator.can_identify-250"><span class="linenos">250</span></a>
|
||||
</span><span id="StarRocks.Generator.can_identify-251"><a href="#StarRocks.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-252"><a href="#StarRocks.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-253"><a href="#StarRocks.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-254"><a href="#StarRocks.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-255"><a href="#StarRocks.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-256"><a href="#StarRocks.Generator.can_identify-256"><span class="linenos">256</span></a>
|
||||
</span><span id="StarRocks.Generator.can_identify-257"><a href="#StarRocks.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-258"><a href="#StarRocks.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-259"><a href="#StarRocks.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> """</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-260"><a href="#StarRocks.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-261"><a href="#StarRocks.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-262"><a href="#StarRocks.Generator.can_identify-262"><span class="linenos">262</span></a>
|
||||
</span><span id="StarRocks.Generator.can_identify-263"><a href="#StarRocks.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-264"><a href="#StarRocks.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="StarRocks.Generator.can_identify-265"><a href="#StarRocks.Generator.can_identify-265"><span class="linenos">265</span></a>
|
||||
</span><span id="StarRocks.Generator.can_identify-266"><a href="#StarRocks.Generator.can_identify-266"><span class="linenos">266</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Checks if text can be identified given an identify option.</p>
|
||||
|
||||
<h6 id="arguments">Arguments:</h6>
|
||||
|
||||
<ul>
|
||||
<li><strong>text:</strong> The text to check.</li>
|
||||
<li><strong>identify:</strong> "always" or <code>True</code>: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Whether or not the given text can be identified.</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
<div class="inherited">
|
||||
<h5>Inherited Members</h5>
|
||||
<dl>
|
||||
|
|
@ -428,6 +489,7 @@ Default: True</li>
|
|||
<dd id="StarRocks.Generator.pragma_sql" class="function"><a href="../generator.html#Generator.pragma_sql">pragma_sql</a></dd>
|
||||
<dd id="StarRocks.Generator.lock_sql" class="function"><a href="../generator.html#Generator.lock_sql">lock_sql</a></dd>
|
||||
<dd id="StarRocks.Generator.literal_sql" class="function"><a href="../generator.html#Generator.literal_sql">literal_sql</a></dd>
|
||||
<dd id="StarRocks.Generator.escape_str" class="function"><a href="../generator.html#Generator.escape_str">escape_str</a></dd>
|
||||
<dd id="StarRocks.Generator.loaddata_sql" class="function"><a href="../generator.html#Generator.loaddata_sql">loaddata_sql</a></dd>
|
||||
<dd id="StarRocks.Generator.null_sql" class="function"><a href="../generator.html#Generator.null_sql">null_sql</a></dd>
|
||||
<dd id="StarRocks.Generator.boolean_sql" class="function"><a href="../generator.html#Generator.boolean_sql">boolean_sql</a></dd>
|
||||
|
|
|
|||
|
|
@ -42,6 +42,9 @@
|
|||
<li>
|
||||
<a class="function" href="#Tableau.Generator.count_sql">count_sql</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="function" href="#Tableau.Generator.can_identify">can_identify</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</li>
|
||||
|
|
@ -175,6 +178,10 @@
|
|||
<div><dt><a href="dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></dt>
|
||||
<dd id="Tableau.get_or_raise" class="function"><a href="dialect.html#Dialect.get_or_raise">get_or_raise</a></dd>
|
||||
<dd id="Tableau.format_time" class="function"><a href="dialect.html#Dialect.format_time">format_time</a></dd>
|
||||
<dd id="Tableau.normalize_identifier" class="function"><a href="dialect.html#Dialect.normalize_identifier">normalize_identifier</a></dd>
|
||||
<dd id="Tableau.case_sensitive" class="function"><a href="dialect.html#Dialect.case_sensitive">case_sensitive</a></dd>
|
||||
<dd id="Tableau.can_identify" class="function"><a href="dialect.html#Dialect.can_identify">can_identify</a></dd>
|
||||
<dd id="Tableau.quote_identifier" class="function"><a href="dialect.html#Dialect.quote_identifier">quote_identifier</a></dd>
|
||||
<dd id="Tableau.parse" class="function"><a href="dialect.html#Dialect.parse">parse</a></dd>
|
||||
<dd id="Tableau.parse_into" class="function"><a href="dialect.html#Dialect.parse_into">parse_into</a></dd>
|
||||
<dd id="Tableau.generate" class="function"><a href="dialect.html#Dialect.generate">generate</a></dd>
|
||||
|
|
@ -309,6 +316,60 @@ Default: True</li>
|
|||
|
||||
|
||||
|
||||
</div>
|
||||
<div id="Tableau.Generator.can_identify" class="classattr">
|
||||
<input id="Tableau.Generator.can_identify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">can_identify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">text</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s1">'safe'</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Tableau.Generator.can_identify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#Tableau.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Tableau.Generator.can_identify-247"><a href="#Tableau.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="Tableau.Generator.can_identify-248"><a href="#Tableau.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="Tableau.Generator.can_identify-249"><a href="#Tableau.Generator.can_identify-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="Tableau.Generator.can_identify-250"><a href="#Tableau.Generator.can_identify-250"><span class="linenos">250</span></a>
|
||||
</span><span id="Tableau.Generator.can_identify-251"><a href="#Tableau.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Tableau.Generator.can_identify-252"><a href="#Tableau.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="Tableau.Generator.can_identify-253"><a href="#Tableau.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="Tableau.Generator.can_identify-254"><a href="#Tableau.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="Tableau.Generator.can_identify-255"><a href="#Tableau.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="Tableau.Generator.can_identify-256"><a href="#Tableau.Generator.can_identify-256"><span class="linenos">256</span></a>
|
||||
</span><span id="Tableau.Generator.can_identify-257"><a href="#Tableau.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Tableau.Generator.can_identify-258"><a href="#Tableau.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="Tableau.Generator.can_identify-259"><a href="#Tableau.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> """</span>
|
||||
</span><span id="Tableau.Generator.can_identify-260"><a href="#Tableau.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="Tableau.Generator.can_identify-261"><a href="#Tableau.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="Tableau.Generator.can_identify-262"><a href="#Tableau.Generator.can_identify-262"><span class="linenos">262</span></a>
|
||||
</span><span id="Tableau.Generator.can_identify-263"><a href="#Tableau.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="Tableau.Generator.can_identify-264"><a href="#Tableau.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="Tableau.Generator.can_identify-265"><a href="#Tableau.Generator.can_identify-265"><span class="linenos">265</span></a>
|
||||
</span><span id="Tableau.Generator.can_identify-266"><a href="#Tableau.Generator.can_identify-266"><span class="linenos">266</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Checks if text can be identified given an identify option.</p>
|
||||
|
||||
<h6 id="arguments">Arguments:</h6>
|
||||
|
||||
<ul>
|
||||
<li><strong>text:</strong> The text to check.</li>
|
||||
<li><strong>identify:</strong> "always" or <code>True</code>: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Whether or not the given text can be identified.</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
<div class="inherited">
|
||||
<h5>Inherited Members</h5>
|
||||
|
|
@ -412,6 +473,7 @@ Default: True</li>
|
|||
<dd id="Tableau.Generator.pragma_sql" class="function"><a href="../generator.html#Generator.pragma_sql">pragma_sql</a></dd>
|
||||
<dd id="Tableau.Generator.lock_sql" class="function"><a href="../generator.html#Generator.lock_sql">lock_sql</a></dd>
|
||||
<dd id="Tableau.Generator.literal_sql" class="function"><a href="../generator.html#Generator.literal_sql">literal_sql</a></dd>
|
||||
<dd id="Tableau.Generator.escape_str" class="function"><a href="../generator.html#Generator.escape_str">escape_str</a></dd>
|
||||
<dd id="Tableau.Generator.loaddata_sql" class="function"><a href="../generator.html#Generator.loaddata_sql">loaddata_sql</a></dd>
|
||||
<dd id="Tableau.Generator.null_sql" class="function"><a href="../generator.html#Generator.null_sql">null_sql</a></dd>
|
||||
<dd id="Tableau.Generator.boolean_sql" class="function"><a href="../generator.html#Generator.boolean_sql">boolean_sql</a></dd>
|
||||
|
|
|
|||
|
|
@ -66,6 +66,9 @@
|
|||
<li>
|
||||
<a class="function" href="#Teradata.Generator.createable_sql">createable_sql</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="function" href="#Teradata.Generator.can_identify">can_identify</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</li>
|
||||
|
|
@ -530,6 +533,10 @@
|
|||
<div><dt><a href="dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></dt>
|
||||
<dd id="Teradata.get_or_raise" class="function"><a href="dialect.html#Dialect.get_or_raise">get_or_raise</a></dd>
|
||||
<dd id="Teradata.format_time" class="function"><a href="dialect.html#Dialect.format_time">format_time</a></dd>
|
||||
<dd id="Teradata.normalize_identifier" class="function"><a href="dialect.html#Dialect.normalize_identifier">normalize_identifier</a></dd>
|
||||
<dd id="Teradata.case_sensitive" class="function"><a href="dialect.html#Dialect.case_sensitive">case_sensitive</a></dd>
|
||||
<dd id="Teradata.can_identify" class="function"><a href="dialect.html#Dialect.can_identify">can_identify</a></dd>
|
||||
<dd id="Teradata.quote_identifier" class="function"><a href="dialect.html#Dialect.quote_identifier">quote_identifier</a></dd>
|
||||
<dd id="Teradata.parse" class="function"><a href="dialect.html#Dialect.parse">parse</a></dd>
|
||||
<dd id="Teradata.parse_into" class="function"><a href="dialect.html#Dialect.parse_into">parse_into</a></dd>
|
||||
<dd id="Teradata.generate" class="function"><a href="dialect.html#Dialect.generate">generate</a></dd>
|
||||
|
|
@ -987,6 +994,60 @@ Default: True</li>
|
|||
|
||||
|
||||
|
||||
</div>
|
||||
<div id="Teradata.Generator.can_identify" class="classattr">
|
||||
<input id="Teradata.Generator.can_identify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">can_identify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">text</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s1">'safe'</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Teradata.Generator.can_identify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#Teradata.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Teradata.Generator.can_identify-247"><a href="#Teradata.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="Teradata.Generator.can_identify-248"><a href="#Teradata.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="Teradata.Generator.can_identify-249"><a href="#Teradata.Generator.can_identify-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="Teradata.Generator.can_identify-250"><a href="#Teradata.Generator.can_identify-250"><span class="linenos">250</span></a>
|
||||
</span><span id="Teradata.Generator.can_identify-251"><a href="#Teradata.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Teradata.Generator.can_identify-252"><a href="#Teradata.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="Teradata.Generator.can_identify-253"><a href="#Teradata.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="Teradata.Generator.can_identify-254"><a href="#Teradata.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="Teradata.Generator.can_identify-255"><a href="#Teradata.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="Teradata.Generator.can_identify-256"><a href="#Teradata.Generator.can_identify-256"><span class="linenos">256</span></a>
|
||||
</span><span id="Teradata.Generator.can_identify-257"><a href="#Teradata.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Teradata.Generator.can_identify-258"><a href="#Teradata.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="Teradata.Generator.can_identify-259"><a href="#Teradata.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> """</span>
|
||||
</span><span id="Teradata.Generator.can_identify-260"><a href="#Teradata.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="Teradata.Generator.can_identify-261"><a href="#Teradata.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="Teradata.Generator.can_identify-262"><a href="#Teradata.Generator.can_identify-262"><span class="linenos">262</span></a>
|
||||
</span><span id="Teradata.Generator.can_identify-263"><a href="#Teradata.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="Teradata.Generator.can_identify-264"><a href="#Teradata.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="Teradata.Generator.can_identify-265"><a href="#Teradata.Generator.can_identify-265"><span class="linenos">265</span></a>
|
||||
</span><span id="Teradata.Generator.can_identify-266"><a href="#Teradata.Generator.can_identify-266"><span class="linenos">266</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Checks if text can be identified given an identify option.</p>
|
||||
|
||||
<h6 id="arguments">Arguments:</h6>
|
||||
|
||||
<ul>
|
||||
<li><strong>text:</strong> The text to check.</li>
|
||||
<li><strong>identify:</strong> "always" or <code>True</code>: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Whether or not the given text can be identified.</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
<div class="inherited">
|
||||
<h5>Inherited Members</h5>
|
||||
|
|
@ -1087,6 +1148,7 @@ Default: True</li>
|
|||
<dd id="Teradata.Generator.pragma_sql" class="function"><a href="../generator.html#Generator.pragma_sql">pragma_sql</a></dd>
|
||||
<dd id="Teradata.Generator.lock_sql" class="function"><a href="../generator.html#Generator.lock_sql">lock_sql</a></dd>
|
||||
<dd id="Teradata.Generator.literal_sql" class="function"><a href="../generator.html#Generator.literal_sql">literal_sql</a></dd>
|
||||
<dd id="Teradata.Generator.escape_str" class="function"><a href="../generator.html#Generator.escape_str">escape_str</a></dd>
|
||||
<dd id="Teradata.Generator.loaddata_sql" class="function"><a href="../generator.html#Generator.loaddata_sql">loaddata_sql</a></dd>
|
||||
<dd id="Teradata.Generator.null_sql" class="function"><a href="../generator.html#Generator.null_sql">null_sql</a></dd>
|
||||
<dd id="Teradata.Generator.boolean_sql" class="function"><a href="../generator.html#Generator.boolean_sql">boolean_sql</a></dd>
|
||||
|
|
|
|||
|
|
@ -36,6 +36,9 @@
|
|||
<li>
|
||||
<a class="class" href="#Trino.Generator">Trino.Generator</a>
|
||||
<ul class="memberlist">
|
||||
<li>
|
||||
<a class="function" href="#Trino.Generator.can_identify">can_identify</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</li>
|
||||
|
|
@ -125,6 +128,10 @@
|
|||
<div><dt><a href="dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></dt>
|
||||
<dd id="Trino.get_or_raise" class="function"><a href="dialect.html#Dialect.get_or_raise">get_or_raise</a></dd>
|
||||
<dd id="Trino.format_time" class="function"><a href="dialect.html#Dialect.format_time">format_time</a></dd>
|
||||
<dd id="Trino.normalize_identifier" class="function"><a href="dialect.html#Dialect.normalize_identifier">normalize_identifier</a></dd>
|
||||
<dd id="Trino.case_sensitive" class="function"><a href="dialect.html#Dialect.case_sensitive">case_sensitive</a></dd>
|
||||
<dd id="Trino.can_identify" class="function"><a href="dialect.html#Dialect.can_identify">can_identify</a></dd>
|
||||
<dd id="Trino.quote_identifier" class="function"><a href="dialect.html#Dialect.quote_identifier">quote_identifier</a></dd>
|
||||
<dd id="Trino.parse" class="function"><a href="dialect.html#Dialect.parse">parse</a></dd>
|
||||
<dd id="Trino.parse_into" class="function"><a href="dialect.html#Dialect.parse_into">parse_into</a></dd>
|
||||
<dd id="Trino.generate" class="function"><a href="dialect.html#Dialect.generate">generate</a></dd>
|
||||
|
|
@ -195,6 +202,60 @@ Default: True</li>
|
|||
</div>
|
||||
|
||||
|
||||
<div id="Trino.Generator.can_identify" class="classattr">
|
||||
<input id="Trino.Generator.can_identify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">can_identify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">text</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s1">'safe'</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Trino.Generator.can_identify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#Trino.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Trino.Generator.can_identify-247"><a href="#Trino.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="Trino.Generator.can_identify-248"><a href="#Trino.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="Trino.Generator.can_identify-249"><a href="#Trino.Generator.can_identify-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="Trino.Generator.can_identify-250"><a href="#Trino.Generator.can_identify-250"><span class="linenos">250</span></a>
|
||||
</span><span id="Trino.Generator.can_identify-251"><a href="#Trino.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Trino.Generator.can_identify-252"><a href="#Trino.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="Trino.Generator.can_identify-253"><a href="#Trino.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="Trino.Generator.can_identify-254"><a href="#Trino.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="Trino.Generator.can_identify-255"><a href="#Trino.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="Trino.Generator.can_identify-256"><a href="#Trino.Generator.can_identify-256"><span class="linenos">256</span></a>
|
||||
</span><span id="Trino.Generator.can_identify-257"><a href="#Trino.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Trino.Generator.can_identify-258"><a href="#Trino.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="Trino.Generator.can_identify-259"><a href="#Trino.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> """</span>
|
||||
</span><span id="Trino.Generator.can_identify-260"><a href="#Trino.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="Trino.Generator.can_identify-261"><a href="#Trino.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="Trino.Generator.can_identify-262"><a href="#Trino.Generator.can_identify-262"><span class="linenos">262</span></a>
|
||||
</span><span id="Trino.Generator.can_identify-263"><a href="#Trino.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="Trino.Generator.can_identify-264"><a href="#Trino.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="Trino.Generator.can_identify-265"><a href="#Trino.Generator.can_identify-265"><span class="linenos">265</span></a>
|
||||
</span><span id="Trino.Generator.can_identify-266"><a href="#Trino.Generator.can_identify-266"><span class="linenos">266</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Checks if text can be identified given an identify option.</p>
|
||||
|
||||
<h6 id="arguments">Arguments:</h6>
|
||||
|
||||
<ul>
|
||||
<li><strong>text:</strong> The text to check.</li>
|
||||
<li><strong>identify:</strong> "always" or <code>True</code>: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Whether or not the given text can be identified.</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
<div class="inherited">
|
||||
<h5>Inherited Members</h5>
|
||||
<dl>
|
||||
|
|
@ -297,6 +358,7 @@ Default: True</li>
|
|||
<dd id="Trino.Generator.pragma_sql" class="function"><a href="../generator.html#Generator.pragma_sql">pragma_sql</a></dd>
|
||||
<dd id="Trino.Generator.lock_sql" class="function"><a href="../generator.html#Generator.lock_sql">lock_sql</a></dd>
|
||||
<dd id="Trino.Generator.literal_sql" class="function"><a href="../generator.html#Generator.literal_sql">literal_sql</a></dd>
|
||||
<dd id="Trino.Generator.escape_str" class="function"><a href="../generator.html#Generator.escape_str">escape_str</a></dd>
|
||||
<dd id="Trino.Generator.loaddata_sql" class="function"><a href="../generator.html#Generator.loaddata_sql">loaddata_sql</a></dd>
|
||||
<dd id="Trino.Generator.null_sql" class="function"><a href="../generator.html#Generator.null_sql">null_sql</a></dd>
|
||||
<dd id="Trino.Generator.boolean_sql" class="function"><a href="../generator.html#Generator.boolean_sql">boolean_sql</a></dd>
|
||||
|
|
|
|||
|
|
@ -60,6 +60,9 @@
|
|||
<li>
|
||||
<a class="function" href="#TSQL.Generator.returnsproperty_sql">returnsproperty_sql</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="function" href="#TSQL.Generator.can_identify">can_identify</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</li>
|
||||
|
|
@ -1030,6 +1033,10 @@
|
|||
<div><dt><a href="dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></dt>
|
||||
<dd id="TSQL.get_or_raise" class="function"><a href="dialect.html#Dialect.get_or_raise">get_or_raise</a></dd>
|
||||
<dd id="TSQL.format_time" class="function"><a href="dialect.html#Dialect.format_time">format_time</a></dd>
|
||||
<dd id="TSQL.normalize_identifier" class="function"><a href="dialect.html#Dialect.normalize_identifier">normalize_identifier</a></dd>
|
||||
<dd id="TSQL.case_sensitive" class="function"><a href="dialect.html#Dialect.case_sensitive">case_sensitive</a></dd>
|
||||
<dd id="TSQL.can_identify" class="function"><a href="dialect.html#Dialect.can_identify">can_identify</a></dd>
|
||||
<dd id="TSQL.quote_identifier" class="function"><a href="dialect.html#Dialect.quote_identifier">quote_identifier</a></dd>
|
||||
<dd id="TSQL.parse" class="function"><a href="dialect.html#Dialect.parse">parse</a></dd>
|
||||
<dd id="TSQL.parse_into" class="function"><a href="dialect.html#Dialect.parse_into">parse_into</a></dd>
|
||||
<dd id="TSQL.generate" class="function"><a href="dialect.html#Dialect.generate">generate</a></dd>
|
||||
|
|
@ -1494,6 +1501,60 @@ Default: True</li>
|
|||
|
||||
|
||||
|
||||
</div>
|
||||
<div id="TSQL.Generator.can_identify" class="classattr">
|
||||
<input id="TSQL.Generator.can_identify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">can_identify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">text</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s1">'safe'</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="TSQL.Generator.can_identify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#TSQL.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="TSQL.Generator.can_identify-247"><a href="#TSQL.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="TSQL.Generator.can_identify-248"><a href="#TSQL.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="TSQL.Generator.can_identify-249"><a href="#TSQL.Generator.can_identify-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="TSQL.Generator.can_identify-250"><a href="#TSQL.Generator.can_identify-250"><span class="linenos">250</span></a>
|
||||
</span><span id="TSQL.Generator.can_identify-251"><a href="#TSQL.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="TSQL.Generator.can_identify-252"><a href="#TSQL.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="TSQL.Generator.can_identify-253"><a href="#TSQL.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="TSQL.Generator.can_identify-254"><a href="#TSQL.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="TSQL.Generator.can_identify-255"><a href="#TSQL.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="TSQL.Generator.can_identify-256"><a href="#TSQL.Generator.can_identify-256"><span class="linenos">256</span></a>
|
||||
</span><span id="TSQL.Generator.can_identify-257"><a href="#TSQL.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="TSQL.Generator.can_identify-258"><a href="#TSQL.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="TSQL.Generator.can_identify-259"><a href="#TSQL.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> """</span>
|
||||
</span><span id="TSQL.Generator.can_identify-260"><a href="#TSQL.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="TSQL.Generator.can_identify-261"><a href="#TSQL.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="TSQL.Generator.can_identify-262"><a href="#TSQL.Generator.can_identify-262"><span class="linenos">262</span></a>
|
||||
</span><span id="TSQL.Generator.can_identify-263"><a href="#TSQL.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="TSQL.Generator.can_identify-264"><a href="#TSQL.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="TSQL.Generator.can_identify-265"><a href="#TSQL.Generator.can_identify-265"><span class="linenos">265</span></a>
|
||||
</span><span id="TSQL.Generator.can_identify-266"><a href="#TSQL.Generator.can_identify-266"><span class="linenos">266</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Checks if text can be identified given an identify option.</p>
|
||||
|
||||
<h6 id="arguments">Arguments:</h6>
|
||||
|
||||
<ul>
|
||||
<li><strong>text:</strong> The text to check.</li>
|
||||
<li><strong>identify:</strong> "always" or <code>True</code>: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Whether or not the given text can be identified.</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
<div class="inherited">
|
||||
<h5>Inherited Members</h5>
|
||||
|
|
@ -1596,6 +1657,7 @@ Default: True</li>
|
|||
<dd id="TSQL.Generator.pragma_sql" class="function"><a href="../generator.html#Generator.pragma_sql">pragma_sql</a></dd>
|
||||
<dd id="TSQL.Generator.lock_sql" class="function"><a href="../generator.html#Generator.lock_sql">lock_sql</a></dd>
|
||||
<dd id="TSQL.Generator.literal_sql" class="function"><a href="../generator.html#Generator.literal_sql">literal_sql</a></dd>
|
||||
<dd id="TSQL.Generator.escape_str" class="function"><a href="../generator.html#Generator.escape_str">escape_str</a></dd>
|
||||
<dd id="TSQL.Generator.loaddata_sql" class="function"><a href="../generator.html#Generator.loaddata_sql">loaddata_sql</a></dd>
|
||||
<dd id="TSQL.Generator.null_sql" class="function"><a href="../generator.html#Generator.null_sql">null_sql</a></dd>
|
||||
<dd id="TSQL.Generator.boolean_sql" class="function"><a href="../generator.html#Generator.boolean_sql">boolean_sql</a></dd>
|
||||
|
|
|
|||
|
|
@ -96,6 +96,9 @@
|
|||
<li>
|
||||
<a class="class" href="#Python.Generator">Python.Generator</a>
|
||||
<ul class="memberlist">
|
||||
<li>
|
||||
<a class="function" href="#Python.Generator.can_identify">can_identify</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
</li>
|
||||
|
|
@ -1542,6 +1545,10 @@
|
|||
<div><dt><a href="../dialects/dialect.html#Dialect">sqlglot.dialects.dialect.Dialect</a></dt>
|
||||
<dd id="Python.get_or_raise" class="function"><a href="../dialects/dialect.html#Dialect.get_or_raise">get_or_raise</a></dd>
|
||||
<dd id="Python.format_time" class="function"><a href="../dialects/dialect.html#Dialect.format_time">format_time</a></dd>
|
||||
<dd id="Python.normalize_identifier" class="function"><a href="../dialects/dialect.html#Dialect.normalize_identifier">normalize_identifier</a></dd>
|
||||
<dd id="Python.case_sensitive" class="function"><a href="../dialects/dialect.html#Dialect.case_sensitive">case_sensitive</a></dd>
|
||||
<dd id="Python.can_identify" class="function"><a href="../dialects/dialect.html#Dialect.can_identify">can_identify</a></dd>
|
||||
<dd id="Python.quote_identifier" class="function"><a href="../dialects/dialect.html#Dialect.quote_identifier">quote_identifier</a></dd>
|
||||
<dd id="Python.parse" class="function"><a href="../dialects/dialect.html#Dialect.parse">parse</a></dd>
|
||||
<dd id="Python.parse_into" class="function"><a href="../dialects/dialect.html#Dialect.parse_into">parse_into</a></dd>
|
||||
<dd id="Python.generate" class="function"><a href="../dialects/dialect.html#Dialect.generate">generate</a></dd>
|
||||
|
|
@ -1661,6 +1668,60 @@ Default: True</li>
|
|||
</div>
|
||||
|
||||
|
||||
<div id="Python.Generator.can_identify" class="classattr">
|
||||
<input id="Python.Generator.can_identify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
|
||||
<div class="attr function">
|
||||
<div class="decorator">@classmethod</div>
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">can_identify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">text</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s1">'safe'</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="Python.Generator.can_identify-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#Python.Generator.can_identify"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Python.Generator.can_identify-247"><a href="#Python.Generator.can_identify-247"><span class="linenos">247</span></a> <span class="nd">@classmethod</span>
|
||||
</span><span id="Python.Generator.can_identify-248"><a href="#Python.Generator.can_identify-248"><span class="linenos">248</span></a> <span class="k">def</span> <span class="nf">can_identify</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">text</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="nb">bool</span> <span class="o">=</span> <span class="s2">"safe"</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
|
||||
</span><span id="Python.Generator.can_identify-249"><a href="#Python.Generator.can_identify-249"><span class="linenos">249</span></a><span class="w"> </span><span class="sd">"""Checks if text can be identified given an identify option.</span>
|
||||
</span><span id="Python.Generator.can_identify-250"><a href="#Python.Generator.can_identify-250"><span class="linenos">250</span></a>
|
||||
</span><span id="Python.Generator.can_identify-251"><a href="#Python.Generator.can_identify-251"><span class="linenos">251</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Python.Generator.can_identify-252"><a href="#Python.Generator.can_identify-252"><span class="linenos">252</span></a><span class="sd"> text: The text to check.</span>
|
||||
</span><span id="Python.Generator.can_identify-253"><a href="#Python.Generator.can_identify-253"><span class="linenos">253</span></a><span class="sd"> identify:</span>
|
||||
</span><span id="Python.Generator.can_identify-254"><a href="#Python.Generator.can_identify-254"><span class="linenos">254</span></a><span class="sd"> "always" or `True`: Always returns true.</span>
|
||||
</span><span id="Python.Generator.can_identify-255"><a href="#Python.Generator.can_identify-255"><span class="linenos">255</span></a><span class="sd"> "safe": True if the identifier is case-insensitive.</span>
|
||||
</span><span id="Python.Generator.can_identify-256"><a href="#Python.Generator.can_identify-256"><span class="linenos">256</span></a>
|
||||
</span><span id="Python.Generator.can_identify-257"><a href="#Python.Generator.can_identify-257"><span class="linenos">257</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Python.Generator.can_identify-258"><a href="#Python.Generator.can_identify-258"><span class="linenos">258</span></a><span class="sd"> Whether or not the given text can be identified.</span>
|
||||
</span><span id="Python.Generator.can_identify-259"><a href="#Python.Generator.can_identify-259"><span class="linenos">259</span></a><span class="sd"> """</span>
|
||||
</span><span id="Python.Generator.can_identify-260"><a href="#Python.Generator.can_identify-260"><span class="linenos">260</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="ow">is</span> <span class="kc">True</span> <span class="ow">or</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"always"</span><span class="p">:</span>
|
||||
</span><span id="Python.Generator.can_identify-261"><a href="#Python.Generator.can_identify-261"><span class="linenos">261</span></a> <span class="k">return</span> <span class="kc">True</span>
|
||||
</span><span id="Python.Generator.can_identify-262"><a href="#Python.Generator.can_identify-262"><span class="linenos">262</span></a>
|
||||
</span><span id="Python.Generator.can_identify-263"><a href="#Python.Generator.can_identify-263"><span class="linenos">263</span></a> <span class="k">if</span> <span class="n">identify</span> <span class="o">==</span> <span class="s2">"safe"</span><span class="p">:</span>
|
||||
</span><span id="Python.Generator.can_identify-264"><a href="#Python.Generator.can_identify-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="ow">not</span> <span class="bp">cls</span><span class="o">.</span><span class="n">case_sensitive</span><span class="p">(</span><span class="n">text</span><span class="p">)</span>
|
||||
</span><span id="Python.Generator.can_identify-265"><a href="#Python.Generator.can_identify-265"><span class="linenos">265</span></a>
|
||||
</span><span id="Python.Generator.can_identify-266"><a href="#Python.Generator.can_identify-266"><span class="linenos">266</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Checks if text can be identified given an identify option.</p>
|
||||
|
||||
<h6 id="arguments">Arguments:</h6>
|
||||
|
||||
<ul>
|
||||
<li><strong>text:</strong> The text to check.</li>
|
||||
<li><strong>identify:</strong> "always" or <code>True</code>: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.</li>
|
||||
</ul>
|
||||
|
||||
<h6 id="returns">Returns:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Whether or not the given text can be identified.</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
|
||||
|
||||
</div>
|
||||
<div class="inherited">
|
||||
<h5>Inherited Members</h5>
|
||||
<dl>
|
||||
|
|
@ -1763,6 +1824,7 @@ Default: True</li>
|
|||
<dd id="Python.Generator.pragma_sql" class="function"><a href="../generator.html#Generator.pragma_sql">pragma_sql</a></dd>
|
||||
<dd id="Python.Generator.lock_sql" class="function"><a href="../generator.html#Generator.lock_sql">lock_sql</a></dd>
|
||||
<dd id="Python.Generator.literal_sql" class="function"><a href="../generator.html#Generator.literal_sql">literal_sql</a></dd>
|
||||
<dd id="Python.Generator.escape_str" class="function"><a href="../generator.html#Generator.escape_str">escape_str</a></dd>
|
||||
<dd id="Python.Generator.loaddata_sql" class="function"><a href="../generator.html#Generator.loaddata_sql">loaddata_sql</a></dd>
|
||||
<dd id="Python.Generator.null_sql" class="function"><a href="../generator.html#Generator.null_sql">null_sql</a></dd>
|
||||
<dd id="Python.Generator.boolean_sql" class="function"><a href="../generator.html#Generator.boolean_sql">boolean_sql</a></dd>
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -93,7 +93,7 @@
|
|||
</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a> <span class="c1"># Reverse the joins so we can remove chains of unused joins</span>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="nb">reversed</span><span class="p">(</span><span class="n">joins</span><span class="p">):</span>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a> <span class="k">if</span> <span class="n">_should_eliminate_join</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">join</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a> <span class="n">join</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">remove_source</span><span class="p">(</span><span class="n">alias</span><span class="p">)</span>
|
||||
|
|
@ -187,7 +187,7 @@
|
|||
</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a><span class="sd"> tuple[list[str], list[str], exp.Expression]:</span>
|
||||
</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a><span class="sd"> Tuple of (source key, join key, remaining predicate)</span>
|
||||
</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="n">on</span> <span class="o">=</span> <span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="n">source_key</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="n">join_key</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
|
|
@ -284,7 +284,7 @@
|
|||
</span><span id="eliminate_joins-33"><a href="#eliminate_joins-33"><span class="linenos">33</span></a>
|
||||
</span><span id="eliminate_joins-34"><a href="#eliminate_joins-34"><span class="linenos">34</span></a> <span class="c1"># Reverse the joins so we can remove chains of unused joins</span>
|
||||
</span><span id="eliminate_joins-35"><a href="#eliminate_joins-35"><span class="linenos">35</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="nb">reversed</span><span class="p">(</span><span class="n">joins</span><span class="p">):</span>
|
||||
</span><span id="eliminate_joins-36"><a href="#eliminate_joins-36"><span class="linenos">36</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="eliminate_joins-36"><a href="#eliminate_joins-36"><span class="linenos">36</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="eliminate_joins-37"><a href="#eliminate_joins-37"><span class="linenos">37</span></a> <span class="k">if</span> <span class="n">_should_eliminate_join</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">join</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="eliminate_joins-38"><a href="#eliminate_joins-38"><span class="linenos">38</span></a> <span class="n">join</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="eliminate_joins-39"><a href="#eliminate_joins-39"><span class="linenos">39</span></a> <span class="n">scope</span><span class="o">.</span><span class="n">remove_source</span><span class="p">(</span><span class="n">alias</span><span class="p">)</span>
|
||||
|
|
@ -345,7 +345,7 @@
|
|||
</span><span id="join_condition-127"><a href="#join_condition-127"><span class="linenos">127</span></a><span class="sd"> tuple[list[str], list[str], exp.Expression]:</span>
|
||||
</span><span id="join_condition-128"><a href="#join_condition-128"><span class="linenos">128</span></a><span class="sd"> Tuple of (source key, join key, remaining predicate)</span>
|
||||
</span><span id="join_condition-129"><a href="#join_condition-129"><span class="linenos">129</span></a><span class="sd"> """</span>
|
||||
</span><span id="join_condition-130"><a href="#join_condition-130"><span class="linenos">130</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="join_condition-130"><a href="#join_condition-130"><span class="linenos">130</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="join_condition-131"><a href="#join_condition-131"><span class="linenos">131</span></a> <span class="n">on</span> <span class="o">=</span> <span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
</span><span id="join_condition-132"><a href="#join_condition-132"><span class="linenos">132</span></a> <span class="n">source_key</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="join_condition-133"><a href="#join_condition-133"><span class="linenos">133</span></a> <span class="n">join_key</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
|
|
|
|||
|
|
@ -209,7 +209,7 @@
|
|||
</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="k">return</span> <span class="kc">False</span>
|
||||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">alias</span> <span class="o">=</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="n">on</span> <span class="o">=</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">on</span><span class="p">:</span>
|
||||
|
|
@ -317,135 +317,130 @@
|
|||
</span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a>
|
||||
</span><span id="L-255"><a href="#L-255"><span class="linenos">255</span></a> <span class="n">new_joins</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a> <span class="n">comma_joins</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">)</span><span class="o">.</span><span class="n">expressions</span><span class="p">[</span><span class="mi">1</span><span class="p">:]</span>
|
||||
</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a> <span class="k">for</span> <span class="n">subquery</span> <span class="ow">in</span> <span class="n">comma_joins</span><span class="p">:</span>
|
||||
</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a> <span class="n">new_joins</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">(</span><span class="n">this</span><span class="o">=</span><span class="n">subquery</span><span class="p">,</span> <span class="n">kind</span><span class="o">=</span><span class="s2">"CROSS"</span><span class="p">))</span>
|
||||
</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">add_source</span><span class="p">(</span><span class="n">subquery</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">subquery</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">])</span>
|
||||
</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a>
|
||||
</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="p">:</span>
|
||||
</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a> <span class="n">new_joins</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">add_source</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">])</span>
|
||||
</span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a>
|
||||
</span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a> <span class="k">if</span> <span class="n">new_joins</span><span class="p">:</span>
|
||||
</span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a> <span class="n">outer_joins</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-268"><a href="#L-268"><span class="linenos">268</span></a>
|
||||
</span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a> <span class="c1"># Maintain the join order</span>
|
||||
</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">):</span>
|
||||
</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a> <span class="n">position</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a> <span class="n">position</span> <span class="o">=</span> <span class="n">outer_joins</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
|
||||
</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a> <span class="n">outer_joins</span><span class="p">[</span><span class="n">position</span><span class="p">:</span><span class="n">position</span><span class="p">]</span> <span class="o">=</span> <span class="n">new_joins</span>
|
||||
</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a>
|
||||
</span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">outer_joins</span><span class="p">)</span>
|
||||
</span><span id="L-277"><a href="#L-277"><span class="linenos">277</span></a>
|
||||
</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a>
|
||||
</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="p">:</span>
|
||||
</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a> <span class="n">new_joins</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">add_source</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">])</span>
|
||||
</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a>
|
||||
</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a> <span class="k">if</span> <span class="n">new_joins</span><span class="p">:</span>
|
||||
</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a> <span class="n">outer_joins</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a>
|
||||
</span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a> <span class="c1"># Maintain the join order</span>
|
||||
</span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">):</span>
|
||||
</span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a> <span class="n">position</span> <span class="o">=</span> <span class="mi">0</span>
|
||||
</span><span id="L-268"><a href="#L-268"><span class="linenos">268</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a> <span class="n">position</span> <span class="o">=</span> <span class="n">outer_joins</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
|
||||
</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a> <span class="n">outer_joins</span><span class="p">[</span><span class="n">position</span><span class="p">:</span><span class="n">position</span><span class="p">]</span> <span class="o">=</span> <span class="n">new_joins</span>
|
||||
</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a>
|
||||
</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="n">outer_joins</span><span class="p">)</span>
|
||||
</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a>
|
||||
</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a>
|
||||
</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a><span class="k">def</span> <span class="nf">_merge_expressions</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-277"><a href="#L-277"><span class="linenos">277</span></a><span class="sd"> Merge projections of inner query into outer query.</span>
|
||||
</span><span id="L-278"><a href="#L-278"><span class="linenos">278</span></a>
|
||||
</span><span id="L-279"><a href="#L-279"><span class="linenos">279</span></a><span class="k">def</span> <span class="nf">_merge_expressions</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">alias</span><span class="p">):</span>
|
||||
</span><span id="L-280"><a href="#L-280"><span class="linenos">280</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-281"><a href="#L-281"><span class="linenos">281</span></a><span class="sd"> Merge projections of inner query into outer query.</span>
|
||||
</span><span id="L-282"><a href="#L-282"><span class="linenos">282</span></a>
|
||||
</span><span id="L-283"><a href="#L-283"><span class="linenos">283</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-284"><a href="#L-284"><span class="linenos">284</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-285"><a href="#L-285"><span class="linenos">285</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-286"><a href="#L-286"><span class="linenos">286</span></a><span class="sd"> alias (str)</span>
|
||||
</span><span id="L-287"><a href="#L-287"><span class="linenos">287</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-288"><a href="#L-288"><span class="linenos">288</span></a> <span class="c1"># Collect all columns that reference the alias of the inner query</span>
|
||||
</span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a> <span class="n">outer_columns</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">list</span><span class="p">)</span>
|
||||
</span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a> <span class="n">outer_columns</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">column</span><span class="p">)</span>
|
||||
</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a>
|
||||
</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a> <span class="c1"># Replace columns with the projection expression in the inner query</span>
|
||||
</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-296"><a href="#L-296"><span class="linenos">296</span></a> <span class="n">projection_name</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-297"><a href="#L-297"><span class="linenos">297</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">projection_name</span><span class="p">:</span>
|
||||
</span><span id="L-298"><a href="#L-298"><span class="linenos">298</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-299"><a href="#L-299"><span class="linenos">299</span></a> <span class="n">columns_to_replace</span> <span class="o">=</span> <span class="n">outer_columns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">projection_name</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-300"><a href="#L-300"><span class="linenos">300</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns_to_replace</span><span class="p">:</span>
|
||||
</span><span id="L-301"><a href="#L-301"><span class="linenos">301</span></a> <span class="n">column</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">unalias</span><span class="p">()</span><span class="o">.</span><span class="n">copy</span><span class="p">())</span>
|
||||
</span><span id="L-302"><a href="#L-302"><span class="linenos">302</span></a>
|
||||
</span><span id="L-279"><a href="#L-279"><span class="linenos">279</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-280"><a href="#L-280"><span class="linenos">280</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-281"><a href="#L-281"><span class="linenos">281</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-282"><a href="#L-282"><span class="linenos">282</span></a><span class="sd"> alias (str)</span>
|
||||
</span><span id="L-283"><a href="#L-283"><span class="linenos">283</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-284"><a href="#L-284"><span class="linenos">284</span></a> <span class="c1"># Collect all columns that reference the alias of the inner query</span>
|
||||
</span><span id="L-285"><a href="#L-285"><span class="linenos">285</span></a> <span class="n">outer_columns</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">list</span><span class="p">)</span>
|
||||
</span><span id="L-286"><a href="#L-286"><span class="linenos">286</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-287"><a href="#L-287"><span class="linenos">287</span></a> <span class="k">if</span> <span class="n">column</span><span class="o">.</span><span class="n">table</span> <span class="o">==</span> <span class="n">alias</span><span class="p">:</span>
|
||||
</span><span id="L-288"><a href="#L-288"><span class="linenos">288</span></a> <span class="n">outer_columns</span><span class="p">[</span><span class="n">column</span><span class="o">.</span><span class="n">name</span><span class="p">]</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">column</span><span class="p">)</span>
|
||||
</span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a>
|
||||
</span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a> <span class="c1"># Replace columns with the projection expression in the inner query</span>
|
||||
</span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a> <span class="n">projection_name</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">projection_name</span><span class="p">:</span>
|
||||
</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a> <span class="n">columns_to_replace</span> <span class="o">=</span> <span class="n">outer_columns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">projection_name</span><span class="p">,</span> <span class="p">[])</span>
|
||||
</span><span id="L-296"><a href="#L-296"><span class="linenos">296</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns_to_replace</span><span class="p">:</span>
|
||||
</span><span id="L-297"><a href="#L-297"><span class="linenos">297</span></a> <span class="n">column</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">unalias</span><span class="p">()</span><span class="o">.</span><span class="n">copy</span><span class="p">())</span>
|
||||
</span><span id="L-298"><a href="#L-298"><span class="linenos">298</span></a>
|
||||
</span><span id="L-299"><a href="#L-299"><span class="linenos">299</span></a>
|
||||
</span><span id="L-300"><a href="#L-300"><span class="linenos">300</span></a><span class="k">def</span> <span class="nf">_merge_where</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">):</span>
|
||||
</span><span id="L-301"><a href="#L-301"><span class="linenos">301</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-302"><a href="#L-302"><span class="linenos">302</span></a><span class="sd"> Merge WHERE clause of inner query into outer query.</span>
|
||||
</span><span id="L-303"><a href="#L-303"><span class="linenos">303</span></a>
|
||||
</span><span id="L-304"><a href="#L-304"><span class="linenos">304</span></a><span class="k">def</span> <span class="nf">_merge_where</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">,</span> <span class="n">from_or_join</span><span class="p">):</span>
|
||||
</span><span id="L-305"><a href="#L-305"><span class="linenos">305</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-306"><a href="#L-306"><span class="linenos">306</span></a><span class="sd"> Merge WHERE clause of inner query into outer query.</span>
|
||||
</span><span id="L-307"><a href="#L-307"><span class="linenos">307</span></a>
|
||||
</span><span id="L-308"><a href="#L-308"><span class="linenos">308</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-309"><a href="#L-309"><span class="linenos">309</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-310"><a href="#L-310"><span class="linenos">310</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-311"><a href="#L-311"><span class="linenos">311</span></a><span class="sd"> from_or_join (exp.From|exp.Join)</span>
|
||||
</span><span id="L-312"><a href="#L-312"><span class="linenos">312</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-313"><a href="#L-313"><span class="linenos">313</span></a> <span class="n">where</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">)</span>
|
||||
</span><span id="L-314"><a href="#L-314"><span class="linenos">314</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">where</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">:</span>
|
||||
</span><span id="L-315"><a href="#L-315"><span class="linenos">315</span></a> <span class="k">return</span>
|
||||
</span><span id="L-316"><a href="#L-316"><span class="linenos">316</span></a>
|
||||
</span><span id="L-317"><a href="#L-317"><span class="linenos">317</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-318"><a href="#L-318"><span class="linenos">318</span></a>
|
||||
</span><span id="L-319"><a href="#L-319"><span class="linenos">319</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-320"><a href="#L-320"><span class="linenos">320</span></a> <span class="c1"># Merge predicates from an outer join to the ON clause</span>
|
||||
</span><span id="L-321"><a href="#L-321"><span class="linenos">321</span></a> <span class="c1"># if it only has columns that are already joined</span>
|
||||
</span><span id="L-322"><a href="#L-322"><span class="linenos">322</span></a> <span class="n">from_</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">)</span>
|
||||
</span><span id="L-323"><a href="#L-323"><span class="linenos">323</span></a> <span class="n">sources</span> <span class="o">=</span> <span class="p">{</span><span class="n">from_</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">}</span> <span class="k">if</span> <span class="n">from_</span> <span class="k">else</span> <span class="p">{}</span>
|
||||
</span><span id="L-324"><a href="#L-324"><span class="linenos">324</span></a>
|
||||
</span><span id="L-325"><a href="#L-325"><span class="linenos">325</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"joins"</span><span class="p">]:</span>
|
||||
</span><span id="L-326"><a href="#L-326"><span class="linenos">326</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-327"><a href="#L-327"><span class="linenos">327</span></a> <span class="n">sources</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
|
||||
</span><span id="L-328"><a href="#L-328"><span class="linenos">328</span></a> <span class="k">if</span> <span class="n">source</span> <span class="o">==</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span>
|
||||
</span><span id="L-329"><a href="#L-329"><span class="linenos">329</span></a> <span class="k">break</span>
|
||||
</span><span id="L-330"><a href="#L-330"><span class="linenos">330</span></a>
|
||||
</span><span id="L-331"><a href="#L-331"><span class="linenos">331</span></a> <span class="k">if</span> <span class="nb">set</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">))</span> <span class="o"><=</span> <span class="n">sources</span><span class="p">:</span>
|
||||
</span><span id="L-332"><a href="#L-332"><span class="linenos">332</span></a> <span class="n">from_or_join</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-333"><a href="#L-333"><span class="linenos">333</span></a> <span class="n">from_or_join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"on"</span><span class="p">,</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">))</span>
|
||||
</span><span id="L-334"><a href="#L-334"><span class="linenos">334</span></a> <span class="k">return</span>
|
||||
</span><span id="L-335"><a href="#L-335"><span class="linenos">335</span></a>
|
||||
</span><span id="L-336"><a href="#L-336"><span class="linenos">336</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-337"><a href="#L-337"><span class="linenos">337</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"where"</span><span class="p">,</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">))</span>
|
||||
</span><span id="L-304"><a href="#L-304"><span class="linenos">304</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-305"><a href="#L-305"><span class="linenos">305</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-306"><a href="#L-306"><span class="linenos">306</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-307"><a href="#L-307"><span class="linenos">307</span></a><span class="sd"> from_or_join (exp.From|exp.Join)</span>
|
||||
</span><span id="L-308"><a href="#L-308"><span class="linenos">308</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-309"><a href="#L-309"><span class="linenos">309</span></a> <span class="n">where</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"where"</span><span class="p">)</span>
|
||||
</span><span id="L-310"><a href="#L-310"><span class="linenos">310</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">where</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">:</span>
|
||||
</span><span id="L-311"><a href="#L-311"><span class="linenos">311</span></a> <span class="k">return</span>
|
||||
</span><span id="L-312"><a href="#L-312"><span class="linenos">312</span></a>
|
||||
</span><span id="L-313"><a href="#L-313"><span class="linenos">313</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span>
|
||||
</span><span id="L-314"><a href="#L-314"><span class="linenos">314</span></a>
|
||||
</span><span id="L-315"><a href="#L-315"><span class="linenos">315</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">from_or_join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-316"><a href="#L-316"><span class="linenos">316</span></a> <span class="c1"># Merge predicates from an outer join to the ON clause</span>
|
||||
</span><span id="L-317"><a href="#L-317"><span class="linenos">317</span></a> <span class="c1"># if it only has columns that are already joined</span>
|
||||
</span><span id="L-318"><a href="#L-318"><span class="linenos">318</span></a> <span class="n">from_</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"from"</span><span class="p">)</span>
|
||||
</span><span id="L-319"><a href="#L-319"><span class="linenos">319</span></a> <span class="n">sources</span> <span class="o">=</span> <span class="p">{</span><span class="n">from_</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">}</span> <span class="k">if</span> <span class="n">from_</span> <span class="k">else</span> <span class="p">{}</span>
|
||||
</span><span id="L-320"><a href="#L-320"><span class="linenos">320</span></a>
|
||||
</span><span id="L-321"><a href="#L-321"><span class="linenos">321</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"joins"</span><span class="p">]:</span>
|
||||
</span><span id="L-322"><a href="#L-322"><span class="linenos">322</span></a> <span class="n">source</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-323"><a href="#L-323"><span class="linenos">323</span></a> <span class="n">sources</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">source</span><span class="p">)</span>
|
||||
</span><span id="L-324"><a href="#L-324"><span class="linenos">324</span></a> <span class="k">if</span> <span class="n">source</span> <span class="o">==</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span>
|
||||
</span><span id="L-325"><a href="#L-325"><span class="linenos">325</span></a> <span class="k">break</span>
|
||||
</span><span id="L-326"><a href="#L-326"><span class="linenos">326</span></a>
|
||||
</span><span id="L-327"><a href="#L-327"><span class="linenos">327</span></a> <span class="k">if</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">)</span> <span class="o"><=</span> <span class="n">sources</span><span class="p">:</span>
|
||||
</span><span id="L-328"><a href="#L-328"><span class="linenos">328</span></a> <span class="n">from_or_join</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-329"><a href="#L-329"><span class="linenos">329</span></a> <span class="n">from_or_join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"on"</span><span class="p">,</span> <span class="n">from_or_join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">))</span>
|
||||
</span><span id="L-330"><a href="#L-330"><span class="linenos">330</span></a> <span class="k">return</span>
|
||||
</span><span id="L-331"><a href="#L-331"><span class="linenos">331</span></a>
|
||||
</span><span id="L-332"><a href="#L-332"><span class="linenos">332</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">where</span><span class="p">(</span><span class="n">where</span><span class="o">.</span><span class="n">this</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-333"><a href="#L-333"><span class="linenos">333</span></a>
|
||||
</span><span id="L-334"><a href="#L-334"><span class="linenos">334</span></a>
|
||||
</span><span id="L-335"><a href="#L-335"><span class="linenos">335</span></a><span class="k">def</span> <span class="nf">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-336"><a href="#L-336"><span class="linenos">336</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-337"><a href="#L-337"><span class="linenos">337</span></a><span class="sd"> Merge ORDER clause of inner query into outer query.</span>
|
||||
</span><span id="L-338"><a href="#L-338"><span class="linenos">338</span></a>
|
||||
</span><span id="L-339"><a href="#L-339"><span class="linenos">339</span></a>
|
||||
</span><span id="L-340"><a href="#L-340"><span class="linenos">340</span></a><span class="k">def</span> <span class="nf">_merge_order</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-341"><a href="#L-341"><span class="linenos">341</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-342"><a href="#L-342"><span class="linenos">342</span></a><span class="sd"> Merge ORDER clause of inner query into outer query.</span>
|
||||
</span><span id="L-343"><a href="#L-343"><span class="linenos">343</span></a>
|
||||
</span><span id="L-344"><a href="#L-344"><span class="linenos">344</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-345"><a href="#L-345"><span class="linenos">345</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-346"><a href="#L-346"><span class="linenos">346</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-347"><a href="#L-347"><span class="linenos">347</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-348"><a href="#L-348"><span class="linenos">348</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-349"><a href="#L-349"><span class="linenos">349</span></a> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-350"><a href="#L-350"><span class="linenos">350</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="p">[</span><span class="s2">"group"</span><span class="p">,</span> <span class="s2">"distinct"</span><span class="p">,</span> <span class="s2">"having"</span><span class="p">,</span> <span class="s2">"order"</span><span class="p">]</span>
|
||||
</span><span id="L-351"><a href="#L-351"><span class="linenos">351</span></a> <span class="p">)</span>
|
||||
</span><span id="L-352"><a href="#L-352"><span class="linenos">352</span></a> <span class="ow">or</span> <span class="nb">len</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span> <span class="o">!=</span> <span class="mi">1</span>
|
||||
</span><span id="L-353"><a href="#L-353"><span class="linenos">353</span></a> <span class="ow">or</span> <span class="nb">any</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">)</span> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span>
|
||||
</span><span id="L-354"><a href="#L-354"><span class="linenos">354</span></a> <span class="p">):</span>
|
||||
</span><span id="L-355"><a href="#L-355"><span class="linenos">355</span></a> <span class="k">return</span>
|
||||
</span><span id="L-356"><a href="#L-356"><span class="linenos">356</span></a>
|
||||
</span><span id="L-357"><a href="#L-357"><span class="linenos">357</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"order"</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"order"</span><span class="p">))</span>
|
||||
</span><span id="L-358"><a href="#L-358"><span class="linenos">358</span></a>
|
||||
</span><span id="L-359"><a href="#L-359"><span class="linenos">359</span></a>
|
||||
</span><span id="L-360"><a href="#L-360"><span class="linenos">360</span></a><span class="k">def</span> <span class="nf">_merge_hints</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-361"><a href="#L-361"><span class="linenos">361</span></a> <span class="n">inner_scope_hint</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">)</span>
|
||||
</span><span id="L-362"><a href="#L-362"><span class="linenos">362</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">inner_scope_hint</span><span class="p">:</span>
|
||||
</span><span id="L-363"><a href="#L-363"><span class="linenos">363</span></a> <span class="k">return</span>
|
||||
</span><span id="L-364"><a href="#L-364"><span class="linenos">364</span></a> <span class="n">outer_scope_hint</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">)</span>
|
||||
</span><span id="L-365"><a href="#L-365"><span class="linenos">365</span></a> <span class="k">if</span> <span class="n">outer_scope_hint</span><span class="p">:</span>
|
||||
</span><span id="L-366"><a href="#L-366"><span class="linenos">366</span></a> <span class="k">for</span> <span class="n">hint_expression</span> <span class="ow">in</span> <span class="n">inner_scope_hint</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-367"><a href="#L-367"><span class="linenos">367</span></a> <span class="n">outer_scope_hint</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"expressions"</span><span class="p">,</span> <span class="n">hint_expression</span><span class="p">)</span>
|
||||
</span><span id="L-368"><a href="#L-368"><span class="linenos">368</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-369"><a href="#L-369"><span class="linenos">369</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">,</span> <span class="n">inner_scope_hint</span><span class="p">)</span>
|
||||
</span><span id="L-339"><a href="#L-339"><span class="linenos">339</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-340"><a href="#L-340"><span class="linenos">340</span></a><span class="sd"> outer_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-341"><a href="#L-341"><span class="linenos">341</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-342"><a href="#L-342"><span class="linenos">342</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-343"><a href="#L-343"><span class="linenos">343</span></a> <span class="k">if</span> <span class="p">(</span>
|
||||
</span><span id="L-344"><a href="#L-344"><span class="linenos">344</span></a> <span class="nb">any</span><span class="p">(</span>
|
||||
</span><span id="L-345"><a href="#L-345"><span class="linenos">345</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="p">[</span><span class="s2">"group"</span><span class="p">,</span> <span class="s2">"distinct"</span><span class="p">,</span> <span class="s2">"having"</span><span class="p">,</span> <span class="s2">"order"</span><span class="p">]</span>
|
||||
</span><span id="L-346"><a href="#L-346"><span class="linenos">346</span></a> <span class="p">)</span>
|
||||
</span><span id="L-347"><a href="#L-347"><span class="linenos">347</span></a> <span class="ow">or</span> <span class="nb">len</span><span class="p">(</span><span class="n">outer_scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">)</span> <span class="o">!=</span> <span class="mi">1</span>
|
||||
</span><span id="L-348"><a href="#L-348"><span class="linenos">348</span></a> <span class="ow">or</span> <span class="nb">any</span><span class="p">(</span><span class="n">expression</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">AggFunc</span><span class="p">)</span> <span class="k">for</span> <span class="n">expression</span> <span class="ow">in</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span>
|
||||
</span><span id="L-349"><a href="#L-349"><span class="linenos">349</span></a> <span class="p">):</span>
|
||||
</span><span id="L-350"><a href="#L-350"><span class="linenos">350</span></a> <span class="k">return</span>
|
||||
</span><span id="L-351"><a href="#L-351"><span class="linenos">351</span></a>
|
||||
</span><span id="L-352"><a href="#L-352"><span class="linenos">352</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"order"</span><span class="p">,</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"order"</span><span class="p">))</span>
|
||||
</span><span id="L-353"><a href="#L-353"><span class="linenos">353</span></a>
|
||||
</span><span id="L-354"><a href="#L-354"><span class="linenos">354</span></a>
|
||||
</span><span id="L-355"><a href="#L-355"><span class="linenos">355</span></a><span class="k">def</span> <span class="nf">_merge_hints</span><span class="p">(</span><span class="n">outer_scope</span><span class="p">,</span> <span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-356"><a href="#L-356"><span class="linenos">356</span></a> <span class="n">inner_scope_hint</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">)</span>
|
||||
</span><span id="L-357"><a href="#L-357"><span class="linenos">357</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">inner_scope_hint</span><span class="p">:</span>
|
||||
</span><span id="L-358"><a href="#L-358"><span class="linenos">358</span></a> <span class="k">return</span>
|
||||
</span><span id="L-359"><a href="#L-359"><span class="linenos">359</span></a> <span class="n">outer_scope_hint</span> <span class="o">=</span> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">)</span>
|
||||
</span><span id="L-360"><a href="#L-360"><span class="linenos">360</span></a> <span class="k">if</span> <span class="n">outer_scope_hint</span><span class="p">:</span>
|
||||
</span><span id="L-361"><a href="#L-361"><span class="linenos">361</span></a> <span class="k">for</span> <span class="n">hint_expression</span> <span class="ow">in</span> <span class="n">inner_scope_hint</span><span class="o">.</span><span class="n">expressions</span><span class="p">:</span>
|
||||
</span><span id="L-362"><a href="#L-362"><span class="linenos">362</span></a> <span class="n">outer_scope_hint</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s2">"expressions"</span><span class="p">,</span> <span class="n">hint_expression</span><span class="p">)</span>
|
||||
</span><span id="L-363"><a href="#L-363"><span class="linenos">363</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-364"><a href="#L-364"><span class="linenos">364</span></a> <span class="n">outer_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"hint"</span><span class="p">,</span> <span class="n">inner_scope_hint</span><span class="p">)</span>
|
||||
</span><span id="L-365"><a href="#L-365"><span class="linenos">365</span></a>
|
||||
</span><span id="L-366"><a href="#L-366"><span class="linenos">366</span></a>
|
||||
</span><span id="L-367"><a href="#L-367"><span class="linenos">367</span></a><span class="k">def</span> <span class="nf">_pop_cte</span><span class="p">(</span><span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-368"><a href="#L-368"><span class="linenos">368</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-369"><a href="#L-369"><span class="linenos">369</span></a><span class="sd"> Remove CTE from the AST.</span>
|
||||
</span><span id="L-370"><a href="#L-370"><span class="linenos">370</span></a>
|
||||
</span><span id="L-371"><a href="#L-371"><span class="linenos">371</span></a>
|
||||
</span><span id="L-372"><a href="#L-372"><span class="linenos">372</span></a><span class="k">def</span> <span class="nf">_pop_cte</span><span class="p">(</span><span class="n">inner_scope</span><span class="p">):</span>
|
||||
</span><span id="L-373"><a href="#L-373"><span class="linenos">373</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-374"><a href="#L-374"><span class="linenos">374</span></a><span class="sd"> Remove CTE from the AST.</span>
|
||||
</span><span id="L-375"><a href="#L-375"><span class="linenos">375</span></a>
|
||||
</span><span id="L-376"><a href="#L-376"><span class="linenos">376</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-377"><a href="#L-377"><span class="linenos">377</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-378"><a href="#L-378"><span class="linenos">378</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-379"><a href="#L-379"><span class="linenos">379</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-380"><a href="#L-380"><span class="linenos">380</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">cte</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-381"><a href="#L-381"><span class="linenos">381</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">with_</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-382"><a href="#L-382"><span class="linenos">382</span></a> <span class="n">with_</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-383"><a href="#L-383"><span class="linenos">383</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-384"><a href="#L-384"><span class="linenos">384</span></a> <span class="n">cte</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-371"><a href="#L-371"><span class="linenos">371</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-372"><a href="#L-372"><span class="linenos">372</span></a><span class="sd"> inner_scope (sqlglot.optimizer.scope.Scope)</span>
|
||||
</span><span id="L-373"><a href="#L-373"><span class="linenos">373</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-374"><a href="#L-374"><span class="linenos">374</span></a> <span class="n">cte</span> <span class="o">=</span> <span class="n">inner_scope</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-375"><a href="#L-375"><span class="linenos">375</span></a> <span class="n">with_</span> <span class="o">=</span> <span class="n">cte</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-376"><a href="#L-376"><span class="linenos">376</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">with_</span><span class="o">.</span><span class="n">expressions</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-377"><a href="#L-377"><span class="linenos">377</span></a> <span class="n">with_</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span><span id="L-378"><a href="#L-378"><span class="linenos">378</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-379"><a href="#L-379"><span class="linenos">379</span></a> <span class="n">cte</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -56,42 +56,33 @@
|
|||
|
||||
<label class="view-source-button" for="mod-normalize_identifiers-view-source"><span>View Source</span></label>
|
||||
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">exp</span>
|
||||
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="kn">from</span> <span class="nn">sqlglot._typing</span> <span class="kn">import</span> <span class="n">E</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">RESOLVES_IDENTIFIERS_AS_UPPERCASE</span><span class="p">,</span> <span class="n">DialectType</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="kn">from</span> <span class="nn">sqlglot._typing</span> <span class="kn">import</span> <span class="n">E</span>
|
||||
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">Dialect</span><span class="p">,</span> <span class="n">DialectType</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a>
|
||||
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a>
|
||||
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="k">def</span> <span class="nf">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="sd"> Normalize all unquoted identifiers to either lower or upper case, depending on</span>
|
||||
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="sd"> the dialect. This essentially makes those identifiers case-insensitive.</span>
|
||||
</span><span id="L-10"><a href="#L-10"><span class="linenos">10</span></a>
|
||||
</span><span id="L-11"><a href="#L-11"><span class="linenos">11</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="sd"> >>> expression = sqlglot.parse_one('SELECT Bar.A AS A FROM "Foo".Bar')</span>
|
||||
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a><span class="sd"> >>> normalize_identifiers(expression).sql()</span>
|
||||
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="sd"> 'SELECT bar.a AS a FROM "Foo".bar'</span>
|
||||
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a>
|
||||
</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a><span class="sd"> expression: The expression to transform.</span>
|
||||
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a><span class="sd"> dialect: The dialect to use in order to decide how to normalize identifiers.</span>
|
||||
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos">22</span></a><span class="sd"> The transformed expression.</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos">23</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos">24</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_normalize</span><span class="p">,</span> <span class="n">dialect</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos">25</span></a>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos">26</span></a>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos">27</span></a><span class="k">def</span> <span class="nf">_normalize</span><span class="p">(</span><span class="n">node</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Expression</span><span class="p">:</span>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos">28</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">node</span><span class="o">.</span><span class="n">quoted</span><span class="p">:</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos">29</span></a> <span class="n">node</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos">30</span></a> <span class="s2">"this"</span><span class="p">,</span>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos">31</span></a> <span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos">32</span></a> <span class="k">if</span> <span class="n">dialect</span> <span class="ow">in</span> <span class="n">RESOLVES_IDENTIFIERS_AS_UPPERCASE</span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos">33</span></a> <span class="k">else</span> <span class="n">node</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">lower</span><span class="p">(),</span>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos">34</span></a> <span class="p">)</span>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos">35</span></a>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos">36</span></a> <span class="k">return</span> <span class="n">node</span>
|
||||
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="k">def</span> <span class="nf">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="sd"> Normalize all unquoted identifiers to either lower or upper case, depending</span>
|
||||
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="sd"> on the dialect. This essentially makes those identifiers case-insensitive.</span>
|
||||
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a>
|
||||
</span><span id="L-10"><a href="#L-10"><span class="linenos">10</span></a><span class="sd"> Note:</span>
|
||||
</span><span id="L-11"><a href="#L-11"><span class="linenos">11</span></a><span class="sd"> Some dialects (e.g. BigQuery) treat identifiers as case-insensitive even</span>
|
||||
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="sd"> when they're quoted, so in these cases all identifiers are normalized.</span>
|
||||
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a>
|
||||
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="sd"> >>> expression = sqlglot.parse_one('SELECT Bar.A AS A FROM "Foo".Bar')</span>
|
||||
</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a><span class="sd"> >>> normalize_identifiers(expression).sql()</span>
|
||||
</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a><span class="sd"> 'SELECT bar.a AS a FROM "Foo".bar'</span>
|
||||
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a>
|
||||
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a><span class="sd"> expression: The expression to transform.</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos">22</span></a><span class="sd"> dialect: The dialect to use in order to decide how to normalize identifiers.</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos">23</span></a>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos">24</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos">25</span></a><span class="sd"> The transformed expression.</span>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos">26</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos">27</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -107,30 +98,41 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#normalize_identifiers"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="normalize_identifiers-7"><a href="#normalize_identifiers-7"><span class="linenos"> 7</span></a><span class="k">def</span> <span class="nf">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="normalize_identifiers-8"><a href="#normalize_identifiers-8"><span class="linenos"> 8</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="normalize_identifiers-9"><a href="#normalize_identifiers-9"><span class="linenos"> 9</span></a><span class="sd"> Normalize all unquoted identifiers to either lower or upper case, depending on</span>
|
||||
</span><span id="normalize_identifiers-10"><a href="#normalize_identifiers-10"><span class="linenos">10</span></a><span class="sd"> the dialect. This essentially makes those identifiers case-insensitive.</span>
|
||||
</span><span id="normalize_identifiers-11"><a href="#normalize_identifiers-11"><span class="linenos">11</span></a>
|
||||
</span><span id="normalize_identifiers-12"><a href="#normalize_identifiers-12"><span class="linenos">12</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="normalize_identifiers-13"><a href="#normalize_identifiers-13"><span class="linenos">13</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="normalize_identifiers-14"><a href="#normalize_identifiers-14"><span class="linenos">14</span></a><span class="sd"> >>> expression = sqlglot.parse_one('SELECT Bar.A AS A FROM "Foo".Bar')</span>
|
||||
</span><span id="normalize_identifiers-15"><a href="#normalize_identifiers-15"><span class="linenos">15</span></a><span class="sd"> >>> normalize_identifiers(expression).sql()</span>
|
||||
</span><span id="normalize_identifiers-16"><a href="#normalize_identifiers-16"><span class="linenos">16</span></a><span class="sd"> 'SELECT bar.a AS a FROM "Foo".bar'</span>
|
||||
</span><span id="normalize_identifiers-17"><a href="#normalize_identifiers-17"><span class="linenos">17</span></a>
|
||||
</span><span id="normalize_identifiers-18"><a href="#normalize_identifiers-18"><span class="linenos">18</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="normalize_identifiers-19"><a href="#normalize_identifiers-19"><span class="linenos">19</span></a><span class="sd"> expression: The expression to transform.</span>
|
||||
</span><span id="normalize_identifiers-20"><a href="#normalize_identifiers-20"><span class="linenos">20</span></a><span class="sd"> dialect: The dialect to use in order to decide how to normalize identifiers.</span>
|
||||
</span><span id="normalize_identifiers-21"><a href="#normalize_identifiers-21"><span class="linenos">21</span></a>
|
||||
</span><span id="normalize_identifiers-22"><a href="#normalize_identifiers-22"><span class="linenos">22</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="normalize_identifiers-23"><a href="#normalize_identifiers-23"><span class="linenos">23</span></a><span class="sd"> The transformed expression.</span>
|
||||
</span><span id="normalize_identifiers-24"><a href="#normalize_identifiers-24"><span class="linenos">24</span></a><span class="sd"> """</span>
|
||||
</span><span id="normalize_identifiers-25"><a href="#normalize_identifiers-25"><span class="linenos">25</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_normalize</span><span class="p">,</span> <span class="n">dialect</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="normalize_identifiers-6"><a href="#normalize_identifiers-6"><span class="linenos"> 6</span></a><span class="k">def</span> <span class="nf">normalize_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="normalize_identifiers-7"><a href="#normalize_identifiers-7"><span class="linenos"> 7</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="normalize_identifiers-8"><a href="#normalize_identifiers-8"><span class="linenos"> 8</span></a><span class="sd"> Normalize all unquoted identifiers to either lower or upper case, depending</span>
|
||||
</span><span id="normalize_identifiers-9"><a href="#normalize_identifiers-9"><span class="linenos"> 9</span></a><span class="sd"> on the dialect. This essentially makes those identifiers case-insensitive.</span>
|
||||
</span><span id="normalize_identifiers-10"><a href="#normalize_identifiers-10"><span class="linenos">10</span></a>
|
||||
</span><span id="normalize_identifiers-11"><a href="#normalize_identifiers-11"><span class="linenos">11</span></a><span class="sd"> Note:</span>
|
||||
</span><span id="normalize_identifiers-12"><a href="#normalize_identifiers-12"><span class="linenos">12</span></a><span class="sd"> Some dialects (e.g. BigQuery) treat identifiers as case-insensitive even</span>
|
||||
</span><span id="normalize_identifiers-13"><a href="#normalize_identifiers-13"><span class="linenos">13</span></a><span class="sd"> when they're quoted, so in these cases all identifiers are normalized.</span>
|
||||
</span><span id="normalize_identifiers-14"><a href="#normalize_identifiers-14"><span class="linenos">14</span></a>
|
||||
</span><span id="normalize_identifiers-15"><a href="#normalize_identifiers-15"><span class="linenos">15</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="normalize_identifiers-16"><a href="#normalize_identifiers-16"><span class="linenos">16</span></a><span class="sd"> >>> import sqlglot</span>
|
||||
</span><span id="normalize_identifiers-17"><a href="#normalize_identifiers-17"><span class="linenos">17</span></a><span class="sd"> >>> expression = sqlglot.parse_one('SELECT Bar.A AS A FROM "Foo".Bar')</span>
|
||||
</span><span id="normalize_identifiers-18"><a href="#normalize_identifiers-18"><span class="linenos">18</span></a><span class="sd"> >>> normalize_identifiers(expression).sql()</span>
|
||||
</span><span id="normalize_identifiers-19"><a href="#normalize_identifiers-19"><span class="linenos">19</span></a><span class="sd"> 'SELECT bar.a AS a FROM "Foo".bar'</span>
|
||||
</span><span id="normalize_identifiers-20"><a href="#normalize_identifiers-20"><span class="linenos">20</span></a>
|
||||
</span><span id="normalize_identifiers-21"><a href="#normalize_identifiers-21"><span class="linenos">21</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="normalize_identifiers-22"><a href="#normalize_identifiers-22"><span class="linenos">22</span></a><span class="sd"> expression: The expression to transform.</span>
|
||||
</span><span id="normalize_identifiers-23"><a href="#normalize_identifiers-23"><span class="linenos">23</span></a><span class="sd"> dialect: The dialect to use in order to decide how to normalize identifiers.</span>
|
||||
</span><span id="normalize_identifiers-24"><a href="#normalize_identifiers-24"><span class="linenos">24</span></a>
|
||||
</span><span id="normalize_identifiers-25"><a href="#normalize_identifiers-25"><span class="linenos">25</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="normalize_identifiers-26"><a href="#normalize_identifiers-26"><span class="linenos">26</span></a><span class="sd"> The transformed expression.</span>
|
||||
</span><span id="normalize_identifiers-27"><a href="#normalize_identifiers-27"><span class="linenos">27</span></a><span class="sd"> """</span>
|
||||
</span><span id="normalize_identifiers-28"><a href="#normalize_identifiers-28"><span class="linenos">28</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
<div class="docstring"><p>Normalize all unquoted identifiers to either lower or upper case, depending on
|
||||
the dialect. This essentially makes those identifiers case-insensitive.</p>
|
||||
<div class="docstring"><p>Normalize all unquoted identifiers to either lower or upper case, depending
|
||||
on the dialect. This essentially makes those identifiers case-insensitive.</p>
|
||||
|
||||
<h6 id="note">Note:</h6>
|
||||
|
||||
<blockquote>
|
||||
<p>Some dialects (e.g. BigQuery) treat identifiers as case-insensitive even
|
||||
when they're quoted, so in these cases all identifiers are normalized.</p>
|
||||
</blockquote>
|
||||
|
||||
<h6 id="example">Example:</h6>
|
||||
|
||||
|
|
|
|||
|
|
@ -65,89 +65,88 @@
|
|||
|
||||
<label class="view-source-button" for="mod-optimize_joins-view-source"><span>View Source</span></label>
|
||||
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">exp</span>
|
||||
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="kn">from</span> <span class="nn">sqlglot.helper</span> <span class="kn">import</span> <span class="n">tsort</span>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="n">JOIN_ATTRS</span> <span class="o">=</span> <span class="p">(</span><span class="s2">"on"</span><span class="p">,</span> <span class="s2">"side"</span><span class="p">,</span> <span class="s2">"kind"</span><span class="p">,</span> <span class="s2">"using"</span><span class="p">,</span> <span class="s2">"method"</span><span class="p">)</span>
|
||||
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a>
|
||||
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a>
|
||||
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="k">def</span> <span class="nf">optimize_joins</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="sd"> Removes cross joins if possible and reorder joins based on predicate dependencies.</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">annotations</span>
|
||||
</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a>
|
||||
</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="kn">import</span> <span class="nn">typing</span> <span class="k">as</span> <span class="nn">t</span>
|
||||
</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a>
|
||||
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">exp</span>
|
||||
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot.helper</span> <span class="kn">import</span> <span class="n">tsort</span>
|
||||
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a>
|
||||
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="n">JOIN_ATTRS</span> <span class="o">=</span> <span class="p">(</span><span class="s2">"on"</span><span class="p">,</span> <span class="s2">"side"</span><span class="p">,</span> <span class="s2">"kind"</span><span class="p">,</span> <span class="s2">"using"</span><span class="p">,</span> <span class="s2">"method"</span><span class="p">)</span>
|
||||
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a>
|
||||
</span><span id="L-10"><a href="#L-10"><span class="linenos">10</span></a>
|
||||
</span><span id="L-11"><a href="#L-11"><span class="linenos">11</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="sd"> >>> from sqlglot import parse_one</span>
|
||||
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="sd"> >>> optimize_joins(parse_one("SELECT * FROM x CROSS JOIN y JOIN z ON x.a = z.a AND y.a = z.a")).sql()</span>
|
||||
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a><span class="sd"> 'SELECT * FROM x JOIN z ON x.a = z.a AND TRUE JOIN y ON y.a = z.a'</span>
|
||||
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a> <span class="n">references</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a> <span class="n">cross_joins</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a>
|
||||
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[]):</span>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos">22</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos">23</span></a>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos">24</span></a> <span class="k">if</span> <span class="n">tables</span><span class="p">:</span>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos">25</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">:</span>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos">26</span></a> <span class="n">references</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">references</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="p">[])</span> <span class="o">+</span> <span class="p">[</span><span class="n">join</span><span class="p">]</span>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos">27</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos">28</span></a> <span class="n">cross_joins</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">name</span><span class="p">,</span> <span class="n">join</span><span class="p">))</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos">29</span></a>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos">30</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">cross_joins</span><span class="p">:</span>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos">31</span></a> <span class="k">for</span> <span class="n">dep</span> <span class="ow">in</span> <span class="n">references</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="p">[]):</span>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos">32</span></a> <span class="n">on</span> <span class="o">=</span> <span class="n">dep</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"on"</span><span class="p">]</span>
|
||||
</span><span id="L-11"><a href="#L-11"><span class="linenos">11</span></a><span class="k">def</span> <span class="nf">optimize_joins</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a><span class="sd"> Removes cross joins if possible and reorder joins based on predicate dependencies.</span>
|
||||
</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a>
|
||||
</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="sd"> >>> from sqlglot import parse_one</span>
|
||||
</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a><span class="sd"> >>> optimize_joins(parse_one("SELECT * FROM x CROSS JOIN y JOIN z ON x.a = z.a AND y.a = z.a")).sql()</span>
|
||||
</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a><span class="sd"> 'SELECT * FROM x JOIN z ON x.a = z.a AND TRUE JOIN y ON y.a = z.a'</span>
|
||||
</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a>
|
||||
</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="L-22"><a href="#L-22"><span class="linenos">22</span></a> <span class="n">references</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-23"><a href="#L-23"><span class="linenos">23</span></a> <span class="n">cross_joins</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-24"><a href="#L-24"><span class="linenos">24</span></a>
|
||||
</span><span id="L-25"><a href="#L-25"><span class="linenos">25</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[]):</span>
|
||||
</span><span id="L-26"><a href="#L-26"><span class="linenos">26</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
</span><span id="L-27"><a href="#L-27"><span class="linenos">27</span></a>
|
||||
</span><span id="L-28"><a href="#L-28"><span class="linenos">28</span></a> <span class="k">if</span> <span class="n">tables</span><span class="p">:</span>
|
||||
</span><span id="L-29"><a href="#L-29"><span class="linenos">29</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">:</span>
|
||||
</span><span id="L-30"><a href="#L-30"><span class="linenos">30</span></a> <span class="n">references</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">references</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="p">[])</span> <span class="o">+</span> <span class="p">[</span><span class="n">join</span><span class="p">]</span>
|
||||
</span><span id="L-31"><a href="#L-31"><span class="linenos">31</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-32"><a href="#L-32"><span class="linenos">32</span></a> <span class="n">cross_joins</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">join</span><span class="p">))</span>
|
||||
</span><span id="L-33"><a href="#L-33"><span class="linenos">33</span></a>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos">34</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">on</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos">35</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">on</span><span class="o">.</span><span class="n">flatten</span><span class="p">():</span>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos">36</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">):</span>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos">37</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos">38</span></a> <span class="n">join</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-39"><a href="#L-39"><span class="linenos">39</span></a>
|
||||
</span><span id="L-40"><a href="#L-40"><span class="linenos">40</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">reorder_joins</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos">41</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos">42</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos">43</span></a>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos">44</span></a>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos">45</span></a><span class="k">def</span> <span class="nf">reorder_joins</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos">46</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos">47</span></a><span class="sd"> Reorder joins by topological sort order based on predicate references.</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos">48</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a> <span class="k">for</span> <span class="n">from_</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">):</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a> <span class="n">head</span> <span class="o">=</span> <span class="n">from_</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">from_</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="p">{</span><span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span> <span class="n">join</span> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">parent</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[])}</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a> <span class="n">dag</span> <span class="o">=</span> <span class="p">{</span><span class="n">head</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span> <span class="p">[]}</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a> <span class="n">dag</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a> <span class="s2">"joins"</span><span class="p">,</span>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="p">[</span><span class="n">joins</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">tsort</span><span class="p">(</span><span class="n">dag</span><span class="p">)</span> <span class="k">if</span> <span class="n">name</span> <span class="o">!=</span> <span class="n">head</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">],</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a> <span class="p">)</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos">63</span></a>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos">64</span></a>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos">65</span></a><span class="k">def</span> <span class="nf">normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos">66</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos">67</span></a><span class="sd"> Remove INNER and OUTER from joins as they are optional.</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos">68</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos">69</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos">70</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">k</span><span class="p">)</span> <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">JOIN_ATTRS</span><span class="p">):</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos">71</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"kind"</span><span class="p">,</span> <span class="s2">"CROSS"</span><span class="p">)</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos">72</span></a>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos">73</span></a> <span class="k">if</span> <span class="n">join</span><span class="o">.</span><span class="n">kind</span> <span class="o">!=</span> <span class="s2">"CROSS"</span><span class="p">:</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos">74</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"kind"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos">76</span></a>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a><span class="k">def</span> <span class="nf">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">,</span> <span class="n">exclude</span><span class="p">):</span>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a> <span class="k">return</span> <span class="p">[</span>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a> <span class="n">name</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">()))</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a> <span class="k">if</span> <span class="n">name</span> <span class="o">!=</span> <span class="n">exclude</span>
|
||||
</span><span id="L-83"><a href="#L-83"><span class="linenos">83</span></a> <span class="p">]</span>
|
||||
</span><span id="L-34"><a href="#L-34"><span class="linenos">34</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">cross_joins</span><span class="p">:</span>
|
||||
</span><span id="L-35"><a href="#L-35"><span class="linenos">35</span></a> <span class="k">for</span> <span class="n">dep</span> <span class="ow">in</span> <span class="n">references</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="p">[]):</span>
|
||||
</span><span id="L-36"><a href="#L-36"><span class="linenos">36</span></a> <span class="n">on</span> <span class="o">=</span> <span class="n">dep</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"on"</span><span class="p">]</span>
|
||||
</span><span id="L-37"><a href="#L-37"><span class="linenos">37</span></a>
|
||||
</span><span id="L-38"><a href="#L-38"><span class="linenos">38</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">on</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
|
||||
</span><span id="L-39"><a href="#L-39"><span class="linenos">39</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">other_table_names</span><span class="p">(</span><span class="n">dep</span><span class="p">))</span> <span class="o"><</span> <span class="mi">2</span><span class="p">:</span>
|
||||
</span><span id="L-40"><a href="#L-40"><span class="linenos">40</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-41"><a href="#L-41"><span class="linenos">41</span></a>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos">42</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">on</span><span class="o">.</span><span class="n">flatten</span><span class="p">():</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos">43</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">):</span>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos">44</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos">45</span></a> <span class="n">join</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos">46</span></a>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos">47</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">reorder_joins</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-48"><a href="#L-48"><span class="linenos">48</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a>
|
||||
</span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a>
|
||||
</span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a><span class="k">def</span> <span class="nf">reorder_joins</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a><span class="sd"> Reorder joins by topological sort order based on predicate references.</span>
|
||||
</span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a> <span class="k">for</span> <span class="n">from_</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">):</span>
|
||||
</span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">from_</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="p">{</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span> <span class="n">join</span> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">parent</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[])}</span>
|
||||
</span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a> <span class="n">dag</span> <span class="o">=</span> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">)</span> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="o">.</span><span class="n">items</span><span class="p">()}</span>
|
||||
</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-61"><a href="#L-61"><span class="linenos">61</span></a> <span class="s2">"joins"</span><span class="p">,</span>
|
||||
</span><span id="L-62"><a href="#L-62"><span class="linenos">62</span></a> <span class="p">[</span><span class="n">joins</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">tsort</span><span class="p">(</span><span class="n">dag</span><span class="p">)</span> <span class="k">if</span> <span class="n">name</span> <span class="o">!=</span> <span class="n">from_</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">],</span>
|
||||
</span><span id="L-63"><a href="#L-63"><span class="linenos">63</span></a> <span class="p">)</span>
|
||||
</span><span id="L-64"><a href="#L-64"><span class="linenos">64</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-65"><a href="#L-65"><span class="linenos">65</span></a>
|
||||
</span><span id="L-66"><a href="#L-66"><span class="linenos">66</span></a>
|
||||
</span><span id="L-67"><a href="#L-67"><span class="linenos">67</span></a><span class="k">def</span> <span class="nf">normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="L-68"><a href="#L-68"><span class="linenos">68</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-69"><a href="#L-69"><span class="linenos">69</span></a><span class="sd"> Remove INNER and OUTER from joins as they are optional.</span>
|
||||
</span><span id="L-70"><a href="#L-70"><span class="linenos">70</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-71"><a href="#L-71"><span class="linenos">71</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="L-72"><a href="#L-72"><span class="linenos">72</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">k</span><span class="p">)</span> <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">JOIN_ATTRS</span><span class="p">):</span>
|
||||
</span><span id="L-73"><a href="#L-73"><span class="linenos">73</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"kind"</span><span class="p">,</span> <span class="s2">"CROSS"</span><span class="p">)</span>
|
||||
</span><span id="L-74"><a href="#L-74"><span class="linenos">74</span></a>
|
||||
</span><span id="L-75"><a href="#L-75"><span class="linenos">75</span></a> <span class="k">if</span> <span class="n">join</span><span class="o">.</span><span class="n">kind</span> <span class="o">!=</span> <span class="s2">"CROSS"</span><span class="p">:</span>
|
||||
</span><span id="L-76"><a href="#L-76"><span class="linenos">76</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"kind"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-77"><a href="#L-77"><span class="linenos">77</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-78"><a href="#L-78"><span class="linenos">78</span></a>
|
||||
</span><span id="L-79"><a href="#L-79"><span class="linenos">79</span></a>
|
||||
</span><span id="L-80"><a href="#L-80"><span class="linenos">80</span></a><span class="k">def</span> <span class="nf">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
||||
</span><span id="L-81"><a href="#L-81"><span class="linenos">81</span></a> <span class="n">on</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span>
|
||||
</span><span id="L-82"><a href="#L-82"><span class="linenos">82</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">on</span><span class="p">,</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">)</span> <span class="k">if</span> <span class="n">on</span> <span class="k">else</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -163,42 +162,45 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#optimize_joins"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="optimize_joins-8"><a href="#optimize_joins-8"><span class="linenos"> 8</span></a><span class="k">def</span> <span class="nf">optimize_joins</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="optimize_joins-9"><a href="#optimize_joins-9"><span class="linenos"> 9</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="optimize_joins-10"><a href="#optimize_joins-10"><span class="linenos">10</span></a><span class="sd"> Removes cross joins if possible and reorder joins based on predicate dependencies.</span>
|
||||
</span><span id="optimize_joins-11"><a href="#optimize_joins-11"><span class="linenos">11</span></a>
|
||||
</span><span id="optimize_joins-12"><a href="#optimize_joins-12"><span class="linenos">12</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="optimize_joins-13"><a href="#optimize_joins-13"><span class="linenos">13</span></a><span class="sd"> >>> from sqlglot import parse_one</span>
|
||||
</span><span id="optimize_joins-14"><a href="#optimize_joins-14"><span class="linenos">14</span></a><span class="sd"> >>> optimize_joins(parse_one("SELECT * FROM x CROSS JOIN y JOIN z ON x.a = z.a AND y.a = z.a")).sql()</span>
|
||||
</span><span id="optimize_joins-15"><a href="#optimize_joins-15"><span class="linenos">15</span></a><span class="sd"> 'SELECT * FROM x JOIN z ON x.a = z.a AND TRUE JOIN y ON y.a = z.a'</span>
|
||||
</span><span id="optimize_joins-16"><a href="#optimize_joins-16"><span class="linenos">16</span></a><span class="sd"> """</span>
|
||||
</span><span id="optimize_joins-17"><a href="#optimize_joins-17"><span class="linenos">17</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="optimize_joins-18"><a href="#optimize_joins-18"><span class="linenos">18</span></a> <span class="n">references</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="optimize_joins-19"><a href="#optimize_joins-19"><span class="linenos">19</span></a> <span class="n">cross_joins</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="optimize_joins-20"><a href="#optimize_joins-20"><span class="linenos">20</span></a>
|
||||
</span><span id="optimize_joins-21"><a href="#optimize_joins-21"><span class="linenos">21</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[]):</span>
|
||||
</span><span id="optimize_joins-22"><a href="#optimize_joins-22"><span class="linenos">22</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="optimize_joins-23"><a href="#optimize_joins-23"><span class="linenos">23</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span>
|
||||
</span><span id="optimize_joins-24"><a href="#optimize_joins-24"><span class="linenos">24</span></a>
|
||||
</span><span id="optimize_joins-25"><a href="#optimize_joins-25"><span class="linenos">25</span></a> <span class="k">if</span> <span class="n">tables</span><span class="p">:</span>
|
||||
</span><span id="optimize_joins-26"><a href="#optimize_joins-26"><span class="linenos">26</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">:</span>
|
||||
</span><span id="optimize_joins-27"><a href="#optimize_joins-27"><span class="linenos">27</span></a> <span class="n">references</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">references</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="p">[])</span> <span class="o">+</span> <span class="p">[</span><span class="n">join</span><span class="p">]</span>
|
||||
</span><span id="optimize_joins-28"><a href="#optimize_joins-28"><span class="linenos">28</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="optimize_joins-29"><a href="#optimize_joins-29"><span class="linenos">29</span></a> <span class="n">cross_joins</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">name</span><span class="p">,</span> <span class="n">join</span><span class="p">))</span>
|
||||
</span><span id="optimize_joins-30"><a href="#optimize_joins-30"><span class="linenos">30</span></a>
|
||||
</span><span id="optimize_joins-31"><a href="#optimize_joins-31"><span class="linenos">31</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">cross_joins</span><span class="p">:</span>
|
||||
</span><span id="optimize_joins-32"><a href="#optimize_joins-32"><span class="linenos">32</span></a> <span class="k">for</span> <span class="n">dep</span> <span class="ow">in</span> <span class="n">references</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="p">[]):</span>
|
||||
</span><span id="optimize_joins-33"><a href="#optimize_joins-33"><span class="linenos">33</span></a> <span class="n">on</span> <span class="o">=</span> <span class="n">dep</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"on"</span><span class="p">]</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="optimize_joins-12"><a href="#optimize_joins-12"><span class="linenos">12</span></a><span class="k">def</span> <span class="nf">optimize_joins</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="optimize_joins-13"><a href="#optimize_joins-13"><span class="linenos">13</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="optimize_joins-14"><a href="#optimize_joins-14"><span class="linenos">14</span></a><span class="sd"> Removes cross joins if possible and reorder joins based on predicate dependencies.</span>
|
||||
</span><span id="optimize_joins-15"><a href="#optimize_joins-15"><span class="linenos">15</span></a>
|
||||
</span><span id="optimize_joins-16"><a href="#optimize_joins-16"><span class="linenos">16</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="optimize_joins-17"><a href="#optimize_joins-17"><span class="linenos">17</span></a><span class="sd"> >>> from sqlglot import parse_one</span>
|
||||
</span><span id="optimize_joins-18"><a href="#optimize_joins-18"><span class="linenos">18</span></a><span class="sd"> >>> optimize_joins(parse_one("SELECT * FROM x CROSS JOIN y JOIN z ON x.a = z.a AND y.a = z.a")).sql()</span>
|
||||
</span><span id="optimize_joins-19"><a href="#optimize_joins-19"><span class="linenos">19</span></a><span class="sd"> 'SELECT * FROM x JOIN z ON x.a = z.a AND TRUE JOIN y ON y.a = z.a'</span>
|
||||
</span><span id="optimize_joins-20"><a href="#optimize_joins-20"><span class="linenos">20</span></a><span class="sd"> """</span>
|
||||
</span><span id="optimize_joins-21"><a href="#optimize_joins-21"><span class="linenos">21</span></a>
|
||||
</span><span id="optimize_joins-22"><a href="#optimize_joins-22"><span class="linenos">22</span></a> <span class="k">for</span> <span class="n">select</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Select</span><span class="p">):</span>
|
||||
</span><span id="optimize_joins-23"><a href="#optimize_joins-23"><span class="linenos">23</span></a> <span class="n">references</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="optimize_joins-24"><a href="#optimize_joins-24"><span class="linenos">24</span></a> <span class="n">cross_joins</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="optimize_joins-25"><a href="#optimize_joins-25"><span class="linenos">25</span></a>
|
||||
</span><span id="optimize_joins-26"><a href="#optimize_joins-26"><span class="linenos">26</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[]):</span>
|
||||
</span><span id="optimize_joins-27"><a href="#optimize_joins-27"><span class="linenos">27</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
</span><span id="optimize_joins-28"><a href="#optimize_joins-28"><span class="linenos">28</span></a>
|
||||
</span><span id="optimize_joins-29"><a href="#optimize_joins-29"><span class="linenos">29</span></a> <span class="k">if</span> <span class="n">tables</span><span class="p">:</span>
|
||||
</span><span id="optimize_joins-30"><a href="#optimize_joins-30"><span class="linenos">30</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">:</span>
|
||||
</span><span id="optimize_joins-31"><a href="#optimize_joins-31"><span class="linenos">31</span></a> <span class="n">references</span><span class="p">[</span><span class="n">table</span><span class="p">]</span> <span class="o">=</span> <span class="n">references</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="p">[])</span> <span class="o">+</span> <span class="p">[</span><span class="n">join</span><span class="p">]</span>
|
||||
</span><span id="optimize_joins-32"><a href="#optimize_joins-32"><span class="linenos">32</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="optimize_joins-33"><a href="#optimize_joins-33"><span class="linenos">33</span></a> <span class="n">cross_joins</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">,</span> <span class="n">join</span><span class="p">))</span>
|
||||
</span><span id="optimize_joins-34"><a href="#optimize_joins-34"><span class="linenos">34</span></a>
|
||||
</span><span id="optimize_joins-35"><a href="#optimize_joins-35"><span class="linenos">35</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">on</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
|
||||
</span><span id="optimize_joins-36"><a href="#optimize_joins-36"><span class="linenos">36</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">on</span><span class="o">.</span><span class="n">flatten</span><span class="p">():</span>
|
||||
</span><span id="optimize_joins-37"><a href="#optimize_joins-37"><span class="linenos">37</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">):</span>
|
||||
</span><span id="optimize_joins-38"><a href="#optimize_joins-38"><span class="linenos">38</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="optimize_joins-39"><a href="#optimize_joins-39"><span class="linenos">39</span></a> <span class="n">join</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="optimize_joins-40"><a href="#optimize_joins-40"><span class="linenos">40</span></a>
|
||||
</span><span id="optimize_joins-41"><a href="#optimize_joins-41"><span class="linenos">41</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">reorder_joins</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="optimize_joins-42"><a href="#optimize_joins-42"><span class="linenos">42</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="optimize_joins-43"><a href="#optimize_joins-43"><span class="linenos">43</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="optimize_joins-35"><a href="#optimize_joins-35"><span class="linenos">35</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">cross_joins</span><span class="p">:</span>
|
||||
</span><span id="optimize_joins-36"><a href="#optimize_joins-36"><span class="linenos">36</span></a> <span class="k">for</span> <span class="n">dep</span> <span class="ow">in</span> <span class="n">references</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="p">[]):</span>
|
||||
</span><span id="optimize_joins-37"><a href="#optimize_joins-37"><span class="linenos">37</span></a> <span class="n">on</span> <span class="o">=</span> <span class="n">dep</span><span class="o">.</span><span class="n">args</span><span class="p">[</span><span class="s2">"on"</span><span class="p">]</span>
|
||||
</span><span id="optimize_joins-38"><a href="#optimize_joins-38"><span class="linenos">38</span></a>
|
||||
</span><span id="optimize_joins-39"><a href="#optimize_joins-39"><span class="linenos">39</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">on</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Connector</span><span class="p">):</span>
|
||||
</span><span id="optimize_joins-40"><a href="#optimize_joins-40"><span class="linenos">40</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">other_table_names</span><span class="p">(</span><span class="n">dep</span><span class="p">))</span> <span class="o"><</span> <span class="mi">2</span><span class="p">:</span>
|
||||
</span><span id="optimize_joins-41"><a href="#optimize_joins-41"><span class="linenos">41</span></a> <span class="k">continue</span>
|
||||
</span><span id="optimize_joins-42"><a href="#optimize_joins-42"><span class="linenos">42</span></a>
|
||||
</span><span id="optimize_joins-43"><a href="#optimize_joins-43"><span class="linenos">43</span></a> <span class="k">for</span> <span class="n">predicate</span> <span class="ow">in</span> <span class="n">on</span><span class="o">.</span><span class="n">flatten</span><span class="p">():</span>
|
||||
</span><span id="optimize_joins-44"><a href="#optimize_joins-44"><span class="linenos">44</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">):</span>
|
||||
</span><span id="optimize_joins-45"><a href="#optimize_joins-45"><span class="linenos">45</span></a> <span class="n">predicate</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">())</span>
|
||||
</span><span id="optimize_joins-46"><a href="#optimize_joins-46"><span class="linenos">46</span></a> <span class="n">join</span><span class="o">.</span><span class="n">on</span><span class="p">(</span><span class="n">predicate</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="optimize_joins-47"><a href="#optimize_joins-47"><span class="linenos">47</span></a>
|
||||
</span><span id="optimize_joins-48"><a href="#optimize_joins-48"><span class="linenos">48</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">reorder_joins</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="optimize_joins-49"><a href="#optimize_joins-49"><span class="linenos">49</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">)</span>
|
||||
</span><span id="optimize_joins-50"><a href="#optimize_joins-50"><span class="linenos">50</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -229,24 +231,19 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#reorder_joins"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="reorder_joins-46"><a href="#reorder_joins-46"><span class="linenos">46</span></a><span class="k">def</span> <span class="nf">reorder_joins</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="reorder_joins-47"><a href="#reorder_joins-47"><span class="linenos">47</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="reorder_joins-48"><a href="#reorder_joins-48"><span class="linenos">48</span></a><span class="sd"> Reorder joins by topological sort order based on predicate references.</span>
|
||||
</span><span id="reorder_joins-49"><a href="#reorder_joins-49"><span class="linenos">49</span></a><span class="sd"> """</span>
|
||||
</span><span id="reorder_joins-50"><a href="#reorder_joins-50"><span class="linenos">50</span></a> <span class="k">for</span> <span class="n">from_</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">):</span>
|
||||
</span><span id="reorder_joins-51"><a href="#reorder_joins-51"><span class="linenos">51</span></a> <span class="n">head</span> <span class="o">=</span> <span class="n">from_</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="reorder_joins-52"><a href="#reorder_joins-52"><span class="linenos">52</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">from_</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="reorder_joins-53"><a href="#reorder_joins-53"><span class="linenos">53</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="p">{</span><span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span> <span class="n">join</span> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">parent</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[])}</span>
|
||||
</span><span id="reorder_joins-54"><a href="#reorder_joins-54"><span class="linenos">54</span></a> <span class="n">dag</span> <span class="o">=</span> <span class="p">{</span><span class="n">head</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span> <span class="p">[]}</span>
|
||||
</span><span id="reorder_joins-55"><a href="#reorder_joins-55"><span class="linenos">55</span></a>
|
||||
</span><span id="reorder_joins-56"><a href="#reorder_joins-56"><span class="linenos">56</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="reorder_joins-57"><a href="#reorder_joins-57"><span class="linenos">57</span></a> <span class="n">dag</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span>
|
||||
</span><span id="reorder_joins-58"><a href="#reorder_joins-58"><span class="linenos">58</span></a>
|
||||
</span><span id="reorder_joins-59"><a href="#reorder_joins-59"><span class="linenos">59</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="reorder_joins-60"><a href="#reorder_joins-60"><span class="linenos">60</span></a> <span class="s2">"joins"</span><span class="p">,</span>
|
||||
</span><span id="reorder_joins-61"><a href="#reorder_joins-61"><span class="linenos">61</span></a> <span class="p">[</span><span class="n">joins</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">tsort</span><span class="p">(</span><span class="n">dag</span><span class="p">)</span> <span class="k">if</span> <span class="n">name</span> <span class="o">!=</span> <span class="n">head</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">],</span>
|
||||
</span><span id="reorder_joins-62"><a href="#reorder_joins-62"><span class="linenos">62</span></a> <span class="p">)</span>
|
||||
</span><span id="reorder_joins-63"><a href="#reorder_joins-63"><span class="linenos">63</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="reorder_joins-53"><a href="#reorder_joins-53"><span class="linenos">53</span></a><span class="k">def</span> <span class="nf">reorder_joins</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="reorder_joins-54"><a href="#reorder_joins-54"><span class="linenos">54</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="reorder_joins-55"><a href="#reorder_joins-55"><span class="linenos">55</span></a><span class="sd"> Reorder joins by topological sort order based on predicate references.</span>
|
||||
</span><span id="reorder_joins-56"><a href="#reorder_joins-56"><span class="linenos">56</span></a><span class="sd"> """</span>
|
||||
</span><span id="reorder_joins-57"><a href="#reorder_joins-57"><span class="linenos">57</span></a> <span class="k">for</span> <span class="n">from_</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">From</span><span class="p">):</span>
|
||||
</span><span id="reorder_joins-58"><a href="#reorder_joins-58"><span class="linenos">58</span></a> <span class="n">parent</span> <span class="o">=</span> <span class="n">from_</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="reorder_joins-59"><a href="#reorder_joins-59"><span class="linenos">59</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="p">{</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">:</span> <span class="n">join</span> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">parent</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">,</span> <span class="p">[])}</span>
|
||||
</span><span id="reorder_joins-60"><a href="#reorder_joins-60"><span class="linenos">60</span></a> <span class="n">dag</span> <span class="o">=</span> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">)</span> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="o">.</span><span class="n">items</span><span class="p">()}</span>
|
||||
</span><span id="reorder_joins-61"><a href="#reorder_joins-61"><span class="linenos">61</span></a> <span class="n">parent</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="reorder_joins-62"><a href="#reorder_joins-62"><span class="linenos">62</span></a> <span class="s2">"joins"</span><span class="p">,</span>
|
||||
</span><span id="reorder_joins-63"><a href="#reorder_joins-63"><span class="linenos">63</span></a> <span class="p">[</span><span class="n">joins</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">tsort</span><span class="p">(</span><span class="n">dag</span><span class="p">)</span> <span class="k">if</span> <span class="n">name</span> <span class="o">!=</span> <span class="n">from_</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">],</span>
|
||||
</span><span id="reorder_joins-64"><a href="#reorder_joins-64"><span class="linenos">64</span></a> <span class="p">)</span>
|
||||
</span><span id="reorder_joins-65"><a href="#reorder_joins-65"><span class="linenos">65</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -266,17 +263,17 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#normalize"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="normalize-66"><a href="#normalize-66"><span class="linenos">66</span></a><span class="k">def</span> <span class="nf">normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="normalize-67"><a href="#normalize-67"><span class="linenos">67</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="normalize-68"><a href="#normalize-68"><span class="linenos">68</span></a><span class="sd"> Remove INNER and OUTER from joins as they are optional.</span>
|
||||
</span><span id="normalize-69"><a href="#normalize-69"><span class="linenos">69</span></a><span class="sd"> """</span>
|
||||
</span><span id="normalize-70"><a href="#normalize-70"><span class="linenos">70</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="normalize-71"><a href="#normalize-71"><span class="linenos">71</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">k</span><span class="p">)</span> <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">JOIN_ATTRS</span><span class="p">):</span>
|
||||
</span><span id="normalize-72"><a href="#normalize-72"><span class="linenos">72</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"kind"</span><span class="p">,</span> <span class="s2">"CROSS"</span><span class="p">)</span>
|
||||
</span><span id="normalize-73"><a href="#normalize-73"><span class="linenos">73</span></a>
|
||||
</span><span id="normalize-74"><a href="#normalize-74"><span class="linenos">74</span></a> <span class="k">if</span> <span class="n">join</span><span class="o">.</span><span class="n">kind</span> <span class="o">!=</span> <span class="s2">"CROSS"</span><span class="p">:</span>
|
||||
</span><span id="normalize-75"><a href="#normalize-75"><span class="linenos">75</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"kind"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="normalize-76"><a href="#normalize-76"><span class="linenos">76</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="normalize-68"><a href="#normalize-68"><span class="linenos">68</span></a><span class="k">def</span> <span class="nf">normalize</span><span class="p">(</span><span class="n">expression</span><span class="p">):</span>
|
||||
</span><span id="normalize-69"><a href="#normalize-69"><span class="linenos">69</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="normalize-70"><a href="#normalize-70"><span class="linenos">70</span></a><span class="sd"> Remove INNER and OUTER from joins as they are optional.</span>
|
||||
</span><span id="normalize-71"><a href="#normalize-71"><span class="linenos">71</span></a><span class="sd"> """</span>
|
||||
</span><span id="normalize-72"><a href="#normalize-72"><span class="linenos">72</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">expression</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">):</span>
|
||||
</span><span id="normalize-73"><a href="#normalize-73"><span class="linenos">73</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">any</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">k</span><span class="p">)</span> <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">JOIN_ATTRS</span><span class="p">):</span>
|
||||
</span><span id="normalize-74"><a href="#normalize-74"><span class="linenos">74</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"kind"</span><span class="p">,</span> <span class="s2">"CROSS"</span><span class="p">)</span>
|
||||
</span><span id="normalize-75"><a href="#normalize-75"><span class="linenos">75</span></a>
|
||||
</span><span id="normalize-76"><a href="#normalize-76"><span class="linenos">76</span></a> <span class="k">if</span> <span class="n">join</span><span class="o">.</span><span class="n">kind</span> <span class="o">!=</span> <span class="s2">"CROSS"</span><span class="p">:</span>
|
||||
</span><span id="normalize-77"><a href="#normalize-77"><span class="linenos">77</span></a> <span class="n">join</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="s2">"kind"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="normalize-78"><a href="#normalize-78"><span class="linenos">78</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -290,18 +287,15 @@
|
|||
<div class="attr function">
|
||||
|
||||
<span class="def">def</span>
|
||||
<span class="name">other_table_names</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">join</span>, </span><span class="param"><span class="n">exclude</span></span><span class="return-annotation">):</span></span>
|
||||
<span class="name">other_table_names</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">join</span><span class="p">:</span> <span class="n"><a href="../expressions.html#Join">sqlglot.expressions.Join</a></span></span><span class="return-annotation">) -> <span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span>:</span></span>
|
||||
|
||||
<label class="view-source-button" for="other_table_names-view-source"><span>View Source</span></label>
|
||||
|
||||
</div>
|
||||
<a class="headerlink" href="#other_table_names"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="other_table_names-79"><a href="#other_table_names-79"><span class="linenos">79</span></a><span class="k">def</span> <span class="nf">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">,</span> <span class="n">exclude</span><span class="p">):</span>
|
||||
</span><span id="other_table_names-80"><a href="#other_table_names-80"><span class="linenos">80</span></a> <span class="k">return</span> <span class="p">[</span>
|
||||
</span><span id="other_table_names-81"><a href="#other_table_names-81"><span class="linenos">81</span></a> <span class="n">name</span>
|
||||
</span><span id="other_table_names-82"><a href="#other_table_names-82"><span class="linenos">82</span></a> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span> <span class="ow">or</span> <span class="n">exp</span><span class="o">.</span><span class="n">true</span><span class="p">()))</span>
|
||||
</span><span id="other_table_names-83"><a href="#other_table_names-83"><span class="linenos">83</span></a> <span class="k">if</span> <span class="n">name</span> <span class="o">!=</span> <span class="n">exclude</span>
|
||||
</span><span id="other_table_names-84"><a href="#other_table_names-84"><span class="linenos">84</span></a> <span class="p">]</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="other_table_names-81"><a href="#other_table_names-81"><span class="linenos">81</span></a><span class="k">def</span> <span class="nf">other_table_names</span><span class="p">(</span><span class="n">join</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]:</span>
|
||||
</span><span id="other_table_names-82"><a href="#other_table_names-82"><span class="linenos">82</span></a> <span class="n">on</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">)</span>
|
||||
</span><span id="other_table_names-83"><a href="#other_table_names-83"><span class="linenos">83</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">on</span><span class="p">,</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">)</span> <span class="k">if</span> <span class="n">on</span> <span class="k">else</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -114,7 +114,7 @@
|
|||
</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="c1"># joins should only pushdown into itself, not to other joins</span>
|
||||
</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a> <span class="c1"># so we limit the selected sources to only itself</span>
|
||||
</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]:</span>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="n">pushdown</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">),</span> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">name</span><span class="p">]},</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a>
|
||||
</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
|
|
@ -166,10 +166,10 @@
|
|||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="n">pushdown_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">))</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">a_tables</span> <span class="o">&=</span> <span class="nb">set</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">))</span>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">a_tables</span> <span class="o">&=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
|
||||
</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a>
|
||||
</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">pushdown_tables</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">a_tables</span><span class="p">)</span>
|
||||
</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a>
|
||||
|
|
@ -220,7 +220,7 @@
|
|||
</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
|
||||
</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">where_condition</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">)</span>
|
||||
</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">:</span>
|
||||
</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">tables</span><span class="p">):</span>
|
||||
</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">source</span> <span class="o">=</span> <span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="ow">or</span> <span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a>
|
||||
</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="c1"># if the predicate is in a where statement we can try to push it down</span>
|
||||
|
|
@ -322,7 +322,7 @@
|
|||
</span><span id="pushdown_predicates-42"><a href="#pushdown_predicates-42"><span class="linenos">42</span></a> <span class="c1"># joins should only pushdown into itself, not to other joins</span>
|
||||
</span><span id="pushdown_predicates-43"><a href="#pushdown_predicates-43"><span class="linenos">43</span></a> <span class="c1"># so we limit the selected sources to only itself</span>
|
||||
</span><span id="pushdown_predicates-44"><a href="#pushdown_predicates-44"><span class="linenos">44</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">select</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"joins"</span><span class="p">)</span> <span class="ow">or</span> <span class="p">[]:</span>
|
||||
</span><span id="pushdown_predicates-45"><a href="#pushdown_predicates-45"><span class="linenos">45</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="pushdown_predicates-45"><a href="#pushdown_predicates-45"><span class="linenos">45</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="pushdown_predicates-46"><a href="#pushdown_predicates-46"><span class="linenos">46</span></a> <span class="n">pushdown</span><span class="p">(</span><span class="n">join</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"on"</span><span class="p">),</span> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">[</span><span class="n">name</span><span class="p">]},</span> <span class="n">scope_ref_count</span><span class="p">)</span>
|
||||
</span><span id="pushdown_predicates-47"><a href="#pushdown_predicates-47"><span class="linenos">47</span></a>
|
||||
</span><span id="pushdown_predicates-48"><a href="#pushdown_predicates-48"><span class="linenos">48</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
|
|
@ -448,10 +448,10 @@
|
|||
</span><span id="pushdown_dnf-94"><a href="#pushdown_dnf-94"><span class="linenos"> 94</span></a> <span class="n">pushdown_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
|
||||
</span><span id="pushdown_dnf-95"><a href="#pushdown_dnf-95"><span class="linenos"> 95</span></a>
|
||||
</span><span id="pushdown_dnf-96"><a href="#pushdown_dnf-96"><span class="linenos"> 96</span></a> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-97"><a href="#pushdown_dnf-97"><span class="linenos"> 97</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">))</span>
|
||||
</span><span id="pushdown_dnf-97"><a href="#pushdown_dnf-97"><span class="linenos"> 97</span></a> <span class="n">a_tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-98"><a href="#pushdown_dnf-98"><span class="linenos"> 98</span></a>
|
||||
</span><span id="pushdown_dnf-99"><a href="#pushdown_dnf-99"><span class="linenos"> 99</span></a> <span class="k">for</span> <span class="n">b</span> <span class="ow">in</span> <span class="n">predicates</span><span class="p">:</span>
|
||||
</span><span id="pushdown_dnf-100"><a href="#pushdown_dnf-100"><span class="linenos">100</span></a> <span class="n">a_tables</span> <span class="o">&=</span> <span class="nb">set</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">))</span>
|
||||
</span><span id="pushdown_dnf-100"><a href="#pushdown_dnf-100"><span class="linenos">100</span></a> <span class="n">a_tables</span> <span class="o">&=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-101"><a href="#pushdown_dnf-101"><span class="linenos">101</span></a>
|
||||
</span><span id="pushdown_dnf-102"><a href="#pushdown_dnf-102"><span class="linenos">102</span></a> <span class="n">pushdown_tables</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">a_tables</span><span class="p">)</span>
|
||||
</span><span id="pushdown_dnf-103"><a href="#pushdown_dnf-103"><span class="linenos">103</span></a>
|
||||
|
|
@ -520,7 +520,7 @@ Additionally, we can't remove predicates from their original form.</p>
|
|||
</span><span id="nodes_for_predicate-148"><a href="#nodes_for_predicate-148"><span class="linenos">148</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">column_table_names</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-149"><a href="#nodes_for_predicate-149"><span class="linenos">149</span></a> <span class="n">where_condition</span> <span class="o">=</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">predicate</span><span class="o">.</span><span class="n">find_ancestor</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">),</span> <span class="n">exp</span><span class="o">.</span><span class="n">Where</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-150"><a href="#nodes_for_predicate-150"><span class="linenos">150</span></a>
|
||||
</span><span id="nodes_for_predicate-151"><a href="#nodes_for_predicate-151"><span class="linenos">151</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">:</span>
|
||||
</span><span id="nodes_for_predicate-151"><a href="#nodes_for_predicate-151"><span class="linenos">151</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">tables</span><span class="p">):</span>
|
||||
</span><span id="nodes_for_predicate-152"><a href="#nodes_for_predicate-152"><span class="linenos">152</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">source</span> <span class="o">=</span> <span class="n">sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table</span><span class="p">)</span> <span class="ow">or</span> <span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="nodes_for_predicate-153"><a href="#nodes_for_predicate-153"><span class="linenos">153</span></a>
|
||||
</span><span id="nodes_for_predicate-154"><a href="#nodes_for_predicate-154"><span class="linenos">154</span></a> <span class="c1"># if the predicate is in a where statement we can try to push it down</span>
|
||||
|
|
|
|||
|
|
@ -87,9 +87,9 @@
|
|||
</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a>
|
||||
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">alias</span><span class="p">,</span> <span class="n">exp</span>
|
||||
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot._typing</span> <span class="kn">import</span> <span class="n">E</span>
|
||||
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">DialectType</span>
|
||||
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">Dialect</span><span class="p">,</span> <span class="n">DialectType</span>
|
||||
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">from</span> <span class="nn">sqlglot.errors</span> <span class="kn">import</span> <span class="n">OptimizeError</span>
|
||||
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="kn">from</span> <span class="nn">sqlglot.helper</span> <span class="kn">import</span> <span class="n">case_sensitive</span><span class="p">,</span> <span class="n">seq_get</span>
|
||||
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="kn">from</span> <span class="nn">sqlglot.helper</span> <span class="kn">import</span> <span class="n">seq_get</span>
|
||||
</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="kn">from</span> <span class="nn">sqlglot.optimizer.scope</span> <span class="kn">import</span> <span class="n">Scope</span><span class="p">,</span> <span class="n">traverse_scope</span><span class="p">,</span> <span class="n">walk_in_scope</span>
|
||||
</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a><span class="kn">from</span> <span class="nn">sqlglot.schema</span> <span class="kn">import</span> <span class="n">Schema</span><span class="p">,</span> <span class="n">ensure_schema</span>
|
||||
</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a>
|
||||
|
|
@ -175,7 +175,7 @@
|
|||
</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a>
|
||||
</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a><span class="k">def</span> <span class="nf">_expand_using</span><span class="p">(</span><span class="n">scope</span><span class="p">,</span> <span class="n">resolver</span><span class="p">):</span>
|
||||
</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">joins</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">scope</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="n">exp</span><span class="o">.</span><span class="n">Join</span><span class="p">))</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">names</span> <span class="o">=</span> <span class="p">{</span><span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="p">}</span>
|
||||
</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">names</span> <span class="o">=</span> <span class="p">{</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="p">}</span>
|
||||
</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">ordered</span> <span class="o">=</span> <span class="p">[</span><span class="n">key</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span> <span class="k">if</span> <span class="n">key</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">names</span><span class="p">]</span>
|
||||
</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a>
|
||||
</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="c1"># Mapping of automatically joined column names to an ordered set of source names (dict).</span>
|
||||
|
|
@ -187,7 +187,7 @@
|
|||
</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">using</span><span class="p">:</span>
|
||||
</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="k">continue</span>
|
||||
</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">join_table</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="n">join_table</span> <span class="o">=</span> <span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span>
|
||||
</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a>
|
||||
</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a>
|
||||
|
|
@ -499,154 +499,144 @@
|
|||
</span><span id="L-417"><a href="#L-417"><span class="linenos">417</span></a>
|
||||
</span><span id="L-418"><a href="#L-418"><span class="linenos">418</span></a><span class="k">def</span> <span class="nf">quote_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="L-419"><a href="#L-419"><span class="linenos">419</span></a><span class="w"> </span><span class="sd">"""Makes sure all identifiers that need to be quoted are quoted."""</span>
|
||||
</span><span id="L-420"><a href="#L-420"><span class="linenos">420</span></a>
|
||||
</span><span id="L-421"><a href="#L-421"><span class="linenos">421</span></a> <span class="k">def</span> <span class="nf">_quote</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="L-422"><a href="#L-422"><span class="linenos">422</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
|
||||
</span><span id="L-423"><a href="#L-423"><span class="linenos">423</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="L-424"><a href="#L-424"><span class="linenos">424</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-425"><a href="#L-425"><span class="linenos">425</span></a> <span class="s2">"quoted"</span><span class="p">,</span>
|
||||
</span><span id="L-426"><a href="#L-426"><span class="linenos">426</span></a> <span class="n">identify</span>
|
||||
</span><span id="L-427"><a href="#L-427"><span class="linenos">427</span></a> <span class="ow">or</span> <span class="n">case_sensitive</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-428"><a href="#L-428"><span class="linenos">428</span></a> <span class="ow">or</span> <span class="ow">not</span> <span class="n">exp</span><span class="o">.</span><span class="n">SAFE_IDENTIFIER_RE</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="n">name</span><span class="p">),</span>
|
||||
</span><span id="L-429"><a href="#L-429"><span class="linenos">429</span></a> <span class="p">)</span>
|
||||
</span><span id="L-430"><a href="#L-430"><span class="linenos">430</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="L-420"><a href="#L-420"><span class="linenos">420</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
|
||||
</span><span id="L-421"><a href="#L-421"><span class="linenos">421</span></a> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">quote_identifier</span><span class="p">,</span> <span class="n">identify</span><span class="o">=</span><span class="n">identify</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span>
|
||||
</span><span id="L-422"><a href="#L-422"><span class="linenos">422</span></a> <span class="p">)</span>
|
||||
</span><span id="L-423"><a href="#L-423"><span class="linenos">423</span></a>
|
||||
</span><span id="L-424"><a href="#L-424"><span class="linenos">424</span></a>
|
||||
</span><span id="L-425"><a href="#L-425"><span class="linenos">425</span></a><span class="k">class</span> <span class="nc">Resolver</span><span class="p">:</span>
|
||||
</span><span id="L-426"><a href="#L-426"><span class="linenos">426</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-427"><a href="#L-427"><span class="linenos">427</span></a><span class="sd"> Helper for resolving columns.</span>
|
||||
</span><span id="L-428"><a href="#L-428"><span class="linenos">428</span></a>
|
||||
</span><span id="L-429"><a href="#L-429"><span class="linenos">429</span></a><span class="sd"> This is a class so we can lazily load some things and easily share them across functions.</span>
|
||||
</span><span id="L-430"><a href="#L-430"><span class="linenos">430</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-431"><a href="#L-431"><span class="linenos">431</span></a>
|
||||
</span><span id="L-432"><a href="#L-432"><span class="linenos">432</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_quote</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="L-433"><a href="#L-433"><span class="linenos">433</span></a>
|
||||
</span><span id="L-434"><a href="#L-434"><span class="linenos">434</span></a>
|
||||
</span><span id="L-435"><a href="#L-435"><span class="linenos">435</span></a><span class="k">class</span> <span class="nc">Resolver</span><span class="p">:</span>
|
||||
</span><span id="L-436"><a href="#L-436"><span class="linenos">436</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-437"><a href="#L-437"><span class="linenos">437</span></a><span class="sd"> Helper for resolving columns.</span>
|
||||
</span><span id="L-438"><a href="#L-438"><span class="linenos">438</span></a>
|
||||
</span><span id="L-439"><a href="#L-439"><span class="linenos">439</span></a><span class="sd"> This is a class so we can lazily load some things and easily share them across functions.</span>
|
||||
</span><span id="L-440"><a href="#L-440"><span class="linenos">440</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-441"><a href="#L-441"><span class="linenos">441</span></a>
|
||||
</span><span id="L-442"><a href="#L-442"><span class="linenos">442</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">infer_schema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">):</span>
|
||||
</span><span id="L-443"><a href="#L-443"><span class="linenos">443</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span> <span class="o">=</span> <span class="n">scope</span>
|
||||
</span><span id="L-444"><a href="#L-444"><span class="linenos">444</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span> <span class="o">=</span> <span class="n">schema</span>
|
||||
</span><span id="L-445"><a href="#L-445"><span class="linenos">445</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-446"><a href="#L-446"><span class="linenos">446</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-447"><a href="#L-447"><span class="linenos">447</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-448"><a href="#L-448"><span class="linenos">448</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_infer_schema</span> <span class="o">=</span> <span class="n">infer_schema</span>
|
||||
</span><span id="L-449"><a href="#L-449"><span class="linenos">449</span></a>
|
||||
</span><span id="L-450"><a href="#L-450"><span class="linenos">450</span></a> <span class="k">def</span> <span class="nf">get_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]:</span>
|
||||
</span><span id="L-451"><a href="#L-451"><span class="linenos">451</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-452"><a href="#L-452"><span class="linenos">452</span></a><span class="sd"> Get the table for a column name.</span>
|
||||
</span><span id="L-432"><a href="#L-432"><span class="linenos">432</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">infer_schema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">):</span>
|
||||
</span><span id="L-433"><a href="#L-433"><span class="linenos">433</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span> <span class="o">=</span> <span class="n">scope</span>
|
||||
</span><span id="L-434"><a href="#L-434"><span class="linenos">434</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span> <span class="o">=</span> <span class="n">schema</span>
|
||||
</span><span id="L-435"><a href="#L-435"><span class="linenos">435</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-436"><a href="#L-436"><span class="linenos">436</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-437"><a href="#L-437"><span class="linenos">437</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-438"><a href="#L-438"><span class="linenos">438</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_infer_schema</span> <span class="o">=</span> <span class="n">infer_schema</span>
|
||||
</span><span id="L-439"><a href="#L-439"><span class="linenos">439</span></a>
|
||||
</span><span id="L-440"><a href="#L-440"><span class="linenos">440</span></a> <span class="k">def</span> <span class="nf">get_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]:</span>
|
||||
</span><span id="L-441"><a href="#L-441"><span class="linenos">441</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-442"><a href="#L-442"><span class="linenos">442</span></a><span class="sd"> Get the table for a column name.</span>
|
||||
</span><span id="L-443"><a href="#L-443"><span class="linenos">443</span></a>
|
||||
</span><span id="L-444"><a href="#L-444"><span class="linenos">444</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-445"><a href="#L-445"><span class="linenos">445</span></a><span class="sd"> column_name: The column name to find the table for.</span>
|
||||
</span><span id="L-446"><a href="#L-446"><span class="linenos">446</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-447"><a href="#L-447"><span class="linenos">447</span></a><span class="sd"> The table name if it can be found/inferred.</span>
|
||||
</span><span id="L-448"><a href="#L-448"><span class="linenos">448</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-449"><a href="#L-449"><span class="linenos">449</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-450"><a href="#L-450"><span class="linenos">450</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_unambiguous_columns</span><span class="p">(</span>
|
||||
</span><span id="L-451"><a href="#L-451"><span class="linenos">451</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span>
|
||||
</span><span id="L-452"><a href="#L-452"><span class="linenos">452</span></a> <span class="p">)</span>
|
||||
</span><span id="L-453"><a href="#L-453"><span class="linenos">453</span></a>
|
||||
</span><span id="L-454"><a href="#L-454"><span class="linenos">454</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-455"><a href="#L-455"><span class="linenos">455</span></a><span class="sd"> column_name: The column name to find the table for.</span>
|
||||
</span><span id="L-456"><a href="#L-456"><span class="linenos">456</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-457"><a href="#L-457"><span class="linenos">457</span></a><span class="sd"> The table name if it can be found/inferred.</span>
|
||||
</span><span id="L-458"><a href="#L-458"><span class="linenos">458</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-459"><a href="#L-459"><span class="linenos">459</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-460"><a href="#L-460"><span class="linenos">460</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_unambiguous_columns</span><span class="p">(</span>
|
||||
</span><span id="L-461"><a href="#L-461"><span class="linenos">461</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span>
|
||||
</span><span id="L-462"><a href="#L-462"><span class="linenos">462</span></a> <span class="p">)</span>
|
||||
</span><span id="L-463"><a href="#L-463"><span class="linenos">463</span></a>
|
||||
</span><span id="L-464"><a href="#L-464"><span class="linenos">464</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column_name</span><span class="p">)</span>
|
||||
</span><span id="L-465"><a href="#L-465"><span class="linenos">465</span></a>
|
||||
</span><span id="L-466"><a href="#L-466"><span class="linenos">466</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_name</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">_infer_schema</span><span class="p">:</span>
|
||||
</span><span id="L-467"><a href="#L-467"><span class="linenos">467</span></a> <span class="n">sources_without_schema</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span>
|
||||
</span><span id="L-468"><a href="#L-468"><span class="linenos">468</span></a> <span class="n">source</span>
|
||||
</span><span id="L-469"><a href="#L-469"><span class="linenos">469</span></a> <span class="k">for</span> <span class="n">source</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
||||
</span><span id="L-470"><a href="#L-470"><span class="linenos">470</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">columns</span> <span class="ow">or</span> <span class="s2">"*"</span> <span class="ow">in</span> <span class="n">columns</span>
|
||||
</span><span id="L-471"><a href="#L-471"><span class="linenos">471</span></a> <span class="p">)</span>
|
||||
</span><span id="L-472"><a href="#L-472"><span class="linenos">472</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">sources_without_schema</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-473"><a href="#L-473"><span class="linenos">473</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">sources_without_schema</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||
</span><span id="L-474"><a href="#L-474"><span class="linenos">474</span></a>
|
||||
</span><span id="L-475"><a href="#L-475"><span class="linenos">475</span></a> <span class="k">if</span> <span class="n">table_name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">:</span>
|
||||
</span><span id="L-476"><a href="#L-476"><span class="linenos">476</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="L-454"><a href="#L-454"><span class="linenos">454</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column_name</span><span class="p">)</span>
|
||||
</span><span id="L-455"><a href="#L-455"><span class="linenos">455</span></a>
|
||||
</span><span id="L-456"><a href="#L-456"><span class="linenos">456</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_name</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">_infer_schema</span><span class="p">:</span>
|
||||
</span><span id="L-457"><a href="#L-457"><span class="linenos">457</span></a> <span class="n">sources_without_schema</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span>
|
||||
</span><span id="L-458"><a href="#L-458"><span class="linenos">458</span></a> <span class="n">source</span>
|
||||
</span><span id="L-459"><a href="#L-459"><span class="linenos">459</span></a> <span class="k">for</span> <span class="n">source</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
||||
</span><span id="L-460"><a href="#L-460"><span class="linenos">460</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">columns</span> <span class="ow">or</span> <span class="s2">"*"</span> <span class="ow">in</span> <span class="n">columns</span>
|
||||
</span><span id="L-461"><a href="#L-461"><span class="linenos">461</span></a> <span class="p">)</span>
|
||||
</span><span id="L-462"><a href="#L-462"><span class="linenos">462</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">sources_without_schema</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-463"><a href="#L-463"><span class="linenos">463</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">sources_without_schema</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||
</span><span id="L-464"><a href="#L-464"><span class="linenos">464</span></a>
|
||||
</span><span id="L-465"><a href="#L-465"><span class="linenos">465</span></a> <span class="k">if</span> <span class="n">table_name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">:</span>
|
||||
</span><span id="L-466"><a href="#L-466"><span class="linenos">466</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="L-467"><a href="#L-467"><span class="linenos">467</span></a>
|
||||
</span><span id="L-468"><a href="#L-468"><span class="linenos">468</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">_</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="L-469"><a href="#L-469"><span class="linenos">469</span></a>
|
||||
</span><span id="L-470"><a href="#L-470"><span class="linenos">470</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subqueryable</span><span class="p">):</span>
|
||||
</span><span id="L-471"><a href="#L-471"><span class="linenos">471</span></a> <span class="k">while</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span> <span class="o">!=</span> <span class="n">table_name</span><span class="p">:</span>
|
||||
</span><span id="L-472"><a href="#L-472"><span class="linenos">472</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-473"><a href="#L-473"><span class="linenos">473</span></a>
|
||||
</span><span id="L-474"><a href="#L-474"><span class="linenos">474</span></a> <span class="n">node_alias</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span>
|
||||
</span><span id="L-475"><a href="#L-475"><span class="linenos">475</span></a> <span class="k">if</span> <span class="n">node_alias</span><span class="p">:</span>
|
||||
</span><span id="L-476"><a href="#L-476"><span class="linenos">476</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">node_alias</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="L-477"><a href="#L-477"><span class="linenos">477</span></a>
|
||||
</span><span id="L-478"><a href="#L-478"><span class="linenos">478</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">_</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="L-478"><a href="#L-478"><span class="linenos">478</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="L-479"><a href="#L-479"><span class="linenos">479</span></a>
|
||||
</span><span id="L-480"><a href="#L-480"><span class="linenos">480</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subqueryable</span><span class="p">):</span>
|
||||
</span><span id="L-481"><a href="#L-481"><span class="linenos">481</span></a> <span class="k">while</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span> <span class="o">!=</span> <span class="n">table_name</span><span class="p">:</span>
|
||||
</span><span id="L-482"><a href="#L-482"><span class="linenos">482</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="L-483"><a href="#L-483"><span class="linenos">483</span></a>
|
||||
</span><span id="L-484"><a href="#L-484"><span class="linenos">484</span></a> <span class="n">node_alias</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span>
|
||||
</span><span id="L-485"><a href="#L-485"><span class="linenos">485</span></a> <span class="k">if</span> <span class="n">node_alias</span><span class="p">:</span>
|
||||
</span><span id="L-486"><a href="#L-486"><span class="linenos">486</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">node_alias</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="L-487"><a href="#L-487"><span class="linenos">487</span></a>
|
||||
</span><span id="L-488"><a href="#L-488"><span class="linenos">488</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="L-489"><a href="#L-489"><span class="linenos">489</span></a>
|
||||
</span><span id="L-490"><a href="#L-490"><span class="linenos">490</span></a> <span class="nd">@property</span>
|
||||
</span><span id="L-491"><a href="#L-491"><span class="linenos">491</span></a> <span class="k">def</span> <span class="nf">all_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-492"><a href="#L-492"><span class="linenos">492</span></a><span class="w"> </span><span class="sd">"""All available columns of all sources in this scope"""</span>
|
||||
</span><span id="L-493"><a href="#L-493"><span class="linenos">493</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-494"><a href="#L-494"><span class="linenos">494</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="L-495"><a href="#L-495"><span class="linenos">495</span></a> <span class="n">column</span> <span class="k">for</span> <span class="n">columns</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span><span class="o">.</span><span class="n">values</span><span class="p">()</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span>
|
||||
</span><span id="L-496"><a href="#L-496"><span class="linenos">496</span></a> <span class="p">}</span>
|
||||
</span><span id="L-497"><a href="#L-497"><span class="linenos">497</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span>
|
||||
</span><span id="L-498"><a href="#L-498"><span class="linenos">498</span></a>
|
||||
</span><span id="L-499"><a href="#L-499"><span class="linenos">499</span></a> <span class="k">def</span> <span class="nf">get_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">only_visible</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
|
||||
</span><span id="L-500"><a href="#L-500"><span class="linenos">500</span></a><span class="w"> </span><span class="sd">"""Resolve the source columns for a given source `name`"""</span>
|
||||
</span><span id="L-501"><a href="#L-501"><span class="linenos">501</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="p">:</span>
|
||||
</span><span id="L-502"><a href="#L-502"><span class="linenos">502</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown table: </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="L-503"><a href="#L-503"><span class="linenos">503</span></a>
|
||||
</span><span id="L-504"><a href="#L-504"><span class="linenos">504</span></a> <span class="n">source</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
|
||||
</span><span id="L-480"><a href="#L-480"><span class="linenos">480</span></a> <span class="nd">@property</span>
|
||||
</span><span id="L-481"><a href="#L-481"><span class="linenos">481</span></a> <span class="k">def</span> <span class="nf">all_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-482"><a href="#L-482"><span class="linenos">482</span></a><span class="w"> </span><span class="sd">"""All available columns of all sources in this scope"""</span>
|
||||
</span><span id="L-483"><a href="#L-483"><span class="linenos">483</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-484"><a href="#L-484"><span class="linenos">484</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="L-485"><a href="#L-485"><span class="linenos">485</span></a> <span class="n">column</span> <span class="k">for</span> <span class="n">columns</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span><span class="o">.</span><span class="n">values</span><span class="p">()</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span>
|
||||
</span><span id="L-486"><a href="#L-486"><span class="linenos">486</span></a> <span class="p">}</span>
|
||||
</span><span id="L-487"><a href="#L-487"><span class="linenos">487</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span>
|
||||
</span><span id="L-488"><a href="#L-488"><span class="linenos">488</span></a>
|
||||
</span><span id="L-489"><a href="#L-489"><span class="linenos">489</span></a> <span class="k">def</span> <span class="nf">get_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">only_visible</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
|
||||
</span><span id="L-490"><a href="#L-490"><span class="linenos">490</span></a><span class="w"> </span><span class="sd">"""Resolve the source columns for a given source `name`"""</span>
|
||||
</span><span id="L-491"><a href="#L-491"><span class="linenos">491</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="p">:</span>
|
||||
</span><span id="L-492"><a href="#L-492"><span class="linenos">492</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown table: </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="L-493"><a href="#L-493"><span class="linenos">493</span></a>
|
||||
</span><span id="L-494"><a href="#L-494"><span class="linenos">494</span></a> <span class="n">source</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
|
||||
</span><span id="L-495"><a href="#L-495"><span class="linenos">495</span></a>
|
||||
</span><span id="L-496"><a href="#L-496"><span class="linenos">496</span></a> <span class="c1"># If referencing a table, return the columns from the schema</span>
|
||||
</span><span id="L-497"><a href="#L-497"><span class="linenos">497</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-498"><a href="#L-498"><span class="linenos">498</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span><span class="o">.</span><span class="n">column_names</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">only_visible</span><span class="p">)</span>
|
||||
</span><span id="L-499"><a href="#L-499"><span class="linenos">499</span></a>
|
||||
</span><span id="L-500"><a href="#L-500"><span class="linenos">500</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">):</span>
|
||||
</span><span id="L-501"><a href="#L-501"><span class="linenos">501</span></a> <span class="k">return</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">alias_column_names</span>
|
||||
</span><span id="L-502"><a href="#L-502"><span class="linenos">502</span></a>
|
||||
</span><span id="L-503"><a href="#L-503"><span class="linenos">503</span></a> <span class="c1"># Otherwise, if referencing another scope, return that scope's named selects</span>
|
||||
</span><span id="L-504"><a href="#L-504"><span class="linenos">504</span></a> <span class="k">return</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span>
|
||||
</span><span id="L-505"><a href="#L-505"><span class="linenos">505</span></a>
|
||||
</span><span id="L-506"><a href="#L-506"><span class="linenos">506</span></a> <span class="c1"># If referencing a table, return the columns from the schema</span>
|
||||
</span><span id="L-507"><a href="#L-507"><span class="linenos">507</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="L-508"><a href="#L-508"><span class="linenos">508</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span><span class="o">.</span><span class="n">column_names</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">only_visible</span><span class="p">)</span>
|
||||
</span><span id="L-509"><a href="#L-509"><span class="linenos">509</span></a>
|
||||
</span><span id="L-510"><a href="#L-510"><span class="linenos">510</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">):</span>
|
||||
</span><span id="L-511"><a href="#L-511"><span class="linenos">511</span></a> <span class="k">return</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">alias_column_names</span>
|
||||
</span><span id="L-512"><a href="#L-512"><span class="linenos">512</span></a>
|
||||
</span><span id="L-513"><a href="#L-513"><span class="linenos">513</span></a> <span class="c1"># Otherwise, if referencing another scope, return that scope's named selects</span>
|
||||
</span><span id="L-514"><a href="#L-514"><span class="linenos">514</span></a> <span class="k">return</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span>
|
||||
</span><span id="L-515"><a href="#L-515"><span class="linenos">515</span></a>
|
||||
</span><span id="L-516"><a href="#L-516"><span class="linenos">516</span></a> <span class="k">def</span> <span class="nf">_get_all_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-517"><a href="#L-517"><span class="linenos">517</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-518"><a href="#L-518"><span class="linenos">518</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="L-519"><a href="#L-519"><span class="linenos">519</span></a> <span class="n">k</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_source_columns</span><span class="p">(</span><span class="n">k</span><span class="p">)</span>
|
||||
</span><span id="L-520"><a href="#L-520"><span class="linenos">520</span></a> <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">lateral_sources</span><span class="p">)</span>
|
||||
</span><span id="L-521"><a href="#L-521"><span class="linenos">521</span></a> <span class="p">}</span>
|
||||
</span><span id="L-522"><a href="#L-522"><span class="linenos">522</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span>
|
||||
</span><span id="L-523"><a href="#L-523"><span class="linenos">523</span></a>
|
||||
</span><span id="L-524"><a href="#L-524"><span class="linenos">524</span></a> <span class="k">def</span> <span class="nf">_get_unambiguous_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">source_columns</span><span class="p">):</span>
|
||||
</span><span id="L-525"><a href="#L-525"><span class="linenos">525</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-526"><a href="#L-526"><span class="linenos">526</span></a><span class="sd"> Find all the unambiguous columns in sources.</span>
|
||||
</span><span id="L-506"><a href="#L-506"><span class="linenos">506</span></a> <span class="k">def</span> <span class="nf">_get_all_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="L-507"><a href="#L-507"><span class="linenos">507</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-508"><a href="#L-508"><span class="linenos">508</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="L-509"><a href="#L-509"><span class="linenos">509</span></a> <span class="n">k</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_source_columns</span><span class="p">(</span><span class="n">k</span><span class="p">)</span>
|
||||
</span><span id="L-510"><a href="#L-510"><span class="linenos">510</span></a> <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">lateral_sources</span><span class="p">)</span>
|
||||
</span><span id="L-511"><a href="#L-511"><span class="linenos">511</span></a> <span class="p">}</span>
|
||||
</span><span id="L-512"><a href="#L-512"><span class="linenos">512</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span>
|
||||
</span><span id="L-513"><a href="#L-513"><span class="linenos">513</span></a>
|
||||
</span><span id="L-514"><a href="#L-514"><span class="linenos">514</span></a> <span class="k">def</span> <span class="nf">_get_unambiguous_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">source_columns</span><span class="p">):</span>
|
||||
</span><span id="L-515"><a href="#L-515"><span class="linenos">515</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-516"><a href="#L-516"><span class="linenos">516</span></a><span class="sd"> Find all the unambiguous columns in sources.</span>
|
||||
</span><span id="L-517"><a href="#L-517"><span class="linenos">517</span></a>
|
||||
</span><span id="L-518"><a href="#L-518"><span class="linenos">518</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-519"><a href="#L-519"><span class="linenos">519</span></a><span class="sd"> source_columns (dict): Mapping of names to source columns</span>
|
||||
</span><span id="L-520"><a href="#L-520"><span class="linenos">520</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-521"><a href="#L-521"><span class="linenos">521</span></a><span class="sd"> dict: Mapping of column name to source name</span>
|
||||
</span><span id="L-522"><a href="#L-522"><span class="linenos">522</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-523"><a href="#L-523"><span class="linenos">523</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source_columns</span><span class="p">:</span>
|
||||
</span><span id="L-524"><a href="#L-524"><span class="linenos">524</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-525"><a href="#L-525"><span class="linenos">525</span></a>
|
||||
</span><span id="L-526"><a href="#L-526"><span class="linenos">526</span></a> <span class="n">source_columns</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">source_columns</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
|
||||
</span><span id="L-527"><a href="#L-527"><span class="linenos">527</span></a>
|
||||
</span><span id="L-528"><a href="#L-528"><span class="linenos">528</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-529"><a href="#L-529"><span class="linenos">529</span></a><span class="sd"> source_columns (dict): Mapping of names to source columns</span>
|
||||
</span><span id="L-530"><a href="#L-530"><span class="linenos">530</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-531"><a href="#L-531"><span class="linenos">531</span></a><span class="sd"> dict: Mapping of column name to source name</span>
|
||||
</span><span id="L-532"><a href="#L-532"><span class="linenos">532</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-533"><a href="#L-533"><span class="linenos">533</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source_columns</span><span class="p">:</span>
|
||||
</span><span id="L-534"><a href="#L-534"><span class="linenos">534</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-535"><a href="#L-535"><span class="linenos">535</span></a>
|
||||
</span><span id="L-536"><a href="#L-536"><span class="linenos">536</span></a> <span class="n">source_columns</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">source_columns</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
|
||||
</span><span id="L-537"><a href="#L-537"><span class="linenos">537</span></a>
|
||||
</span><span id="L-538"><a href="#L-538"><span class="linenos">538</span></a> <span class="n">first_table</span><span class="p">,</span> <span class="n">first_columns</span> <span class="o">=</span> <span class="n">source_columns</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||
</span><span id="L-539"><a href="#L-539"><span class="linenos">539</span></a> <span class="n">unambiguous_columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">col</span><span class="p">:</span> <span class="n">first_table</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">first_columns</span><span class="p">)}</span>
|
||||
</span><span id="L-540"><a href="#L-540"><span class="linenos">540</span></a> <span class="n">all_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">unambiguous_columns</span><span class="p">)</span>
|
||||
</span><span id="L-541"><a href="#L-541"><span class="linenos">541</span></a>
|
||||
</span><span id="L-542"><a href="#L-542"><span class="linenos">542</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="n">source_columns</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
||||
</span><span id="L-543"><a href="#L-543"><span class="linenos">543</span></a> <span class="n">unique</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-544"><a href="#L-544"><span class="linenos">544</span></a> <span class="n">ambiguous</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_columns</span><span class="p">)</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="n">unique</span><span class="p">)</span>
|
||||
</span><span id="L-545"><a href="#L-545"><span class="linenos">545</span></a> <span class="n">all_columns</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-546"><a href="#L-546"><span class="linenos">546</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">ambiguous</span><span class="p">:</span>
|
||||
</span><span id="L-547"><a href="#L-547"><span class="linenos">547</span></a> <span class="n">unambiguous_columns</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-548"><a href="#L-548"><span class="linenos">548</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">unique</span><span class="o">.</span><span class="n">difference</span><span class="p">(</span><span class="n">ambiguous</span><span class="p">):</span>
|
||||
</span><span id="L-549"><a href="#L-549"><span class="linenos">549</span></a> <span class="n">unambiguous_columns</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="L-550"><a href="#L-550"><span class="linenos">550</span></a>
|
||||
</span><span id="L-551"><a href="#L-551"><span class="linenos">551</span></a> <span class="k">return</span> <span class="n">unambiguous_columns</span>
|
||||
</span><span id="L-552"><a href="#L-552"><span class="linenos">552</span></a>
|
||||
</span><span id="L-553"><a href="#L-553"><span class="linenos">553</span></a> <span class="nd">@staticmethod</span>
|
||||
</span><span id="L-554"><a href="#L-554"><span class="linenos">554</span></a> <span class="k">def</span> <span class="nf">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">):</span>
|
||||
</span><span id="L-555"><a href="#L-555"><span class="linenos">555</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-556"><a href="#L-556"><span class="linenos">556</span></a><span class="sd"> Find the unique columns in a list of columns.</span>
|
||||
</span><span id="L-557"><a href="#L-557"><span class="linenos">557</span></a>
|
||||
</span><span id="L-558"><a href="#L-558"><span class="linenos">558</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="L-559"><a href="#L-559"><span class="linenos">559</span></a><span class="sd"> >>> sorted(Resolver._find_unique_columns(["a", "b", "b", "c"]))</span>
|
||||
</span><span id="L-560"><a href="#L-560"><span class="linenos">560</span></a><span class="sd"> ['a', 'c']</span>
|
||||
</span><span id="L-561"><a href="#L-561"><span class="linenos">561</span></a>
|
||||
</span><span id="L-562"><a href="#L-562"><span class="linenos">562</span></a><span class="sd"> This is necessary because duplicate column names are ambiguous.</span>
|
||||
</span><span id="L-563"><a href="#L-563"><span class="linenos">563</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-564"><a href="#L-564"><span class="linenos">564</span></a> <span class="n">counts</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-565"><a href="#L-565"><span class="linenos">565</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-566"><a href="#L-566"><span class="linenos">566</span></a> <span class="n">counts</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">counts</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
|
||||
</span><span id="L-567"><a href="#L-567"><span class="linenos">567</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span><span class="p">,</span> <span class="n">count</span> <span class="ow">in</span> <span class="n">counts</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">count</span> <span class="o">==</span> <span class="mi">1</span><span class="p">}</span>
|
||||
</span><span id="L-528"><a href="#L-528"><span class="linenos">528</span></a> <span class="n">first_table</span><span class="p">,</span> <span class="n">first_columns</span> <span class="o">=</span> <span class="n">source_columns</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||
</span><span id="L-529"><a href="#L-529"><span class="linenos">529</span></a> <span class="n">unambiguous_columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">col</span><span class="p">:</span> <span class="n">first_table</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">first_columns</span><span class="p">)}</span>
|
||||
</span><span id="L-530"><a href="#L-530"><span class="linenos">530</span></a> <span class="n">all_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">unambiguous_columns</span><span class="p">)</span>
|
||||
</span><span id="L-531"><a href="#L-531"><span class="linenos">531</span></a>
|
||||
</span><span id="L-532"><a href="#L-532"><span class="linenos">532</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="n">source_columns</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
||||
</span><span id="L-533"><a href="#L-533"><span class="linenos">533</span></a> <span class="n">unique</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-534"><a href="#L-534"><span class="linenos">534</span></a> <span class="n">ambiguous</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_columns</span><span class="p">)</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="n">unique</span><span class="p">)</span>
|
||||
</span><span id="L-535"><a href="#L-535"><span class="linenos">535</span></a> <span class="n">all_columns</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="L-536"><a href="#L-536"><span class="linenos">536</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">ambiguous</span><span class="p">:</span>
|
||||
</span><span id="L-537"><a href="#L-537"><span class="linenos">537</span></a> <span class="n">unambiguous_columns</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="L-538"><a href="#L-538"><span class="linenos">538</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">unique</span><span class="o">.</span><span class="n">difference</span><span class="p">(</span><span class="n">ambiguous</span><span class="p">):</span>
|
||||
</span><span id="L-539"><a href="#L-539"><span class="linenos">539</span></a> <span class="n">unambiguous_columns</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="L-540"><a href="#L-540"><span class="linenos">540</span></a>
|
||||
</span><span id="L-541"><a href="#L-541"><span class="linenos">541</span></a> <span class="k">return</span> <span class="n">unambiguous_columns</span>
|
||||
</span><span id="L-542"><a href="#L-542"><span class="linenos">542</span></a>
|
||||
</span><span id="L-543"><a href="#L-543"><span class="linenos">543</span></a> <span class="nd">@staticmethod</span>
|
||||
</span><span id="L-544"><a href="#L-544"><span class="linenos">544</span></a> <span class="k">def</span> <span class="nf">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">):</span>
|
||||
</span><span id="L-545"><a href="#L-545"><span class="linenos">545</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-546"><a href="#L-546"><span class="linenos">546</span></a><span class="sd"> Find the unique columns in a list of columns.</span>
|
||||
</span><span id="L-547"><a href="#L-547"><span class="linenos">547</span></a>
|
||||
</span><span id="L-548"><a href="#L-548"><span class="linenos">548</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="L-549"><a href="#L-549"><span class="linenos">549</span></a><span class="sd"> >>> sorted(Resolver._find_unique_columns(["a", "b", "b", "c"]))</span>
|
||||
</span><span id="L-550"><a href="#L-550"><span class="linenos">550</span></a><span class="sd"> ['a', 'c']</span>
|
||||
</span><span id="L-551"><a href="#L-551"><span class="linenos">551</span></a>
|
||||
</span><span id="L-552"><a href="#L-552"><span class="linenos">552</span></a><span class="sd"> This is necessary because duplicate column names are ambiguous.</span>
|
||||
</span><span id="L-553"><a href="#L-553"><span class="linenos">553</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-554"><a href="#L-554"><span class="linenos">554</span></a> <span class="n">counts</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="L-555"><a href="#L-555"><span class="linenos">555</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="L-556"><a href="#L-556"><span class="linenos">556</span></a> <span class="n">counts</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">counts</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
|
||||
</span><span id="L-557"><a href="#L-557"><span class="linenos">557</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span><span class="p">,</span> <span class="n">count</span> <span class="ow">in</span> <span class="n">counts</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">count</span> <span class="o">==</span> <span class="mi">1</span><span class="p">}</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -793,19 +783,9 @@
|
|||
<a class="headerlink" href="#quote_identifiers"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="quote_identifiers-419"><a href="#quote_identifiers-419"><span class="linenos">419</span></a><span class="k">def</span> <span class="nf">quote_identifiers</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="n">identify</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="quote_identifiers-420"><a href="#quote_identifiers-420"><span class="linenos">420</span></a><span class="w"> </span><span class="sd">"""Makes sure all identifiers that need to be quoted are quoted."""</span>
|
||||
</span><span id="quote_identifiers-421"><a href="#quote_identifiers-421"><span class="linenos">421</span></a>
|
||||
</span><span id="quote_identifiers-422"><a href="#quote_identifiers-422"><span class="linenos">422</span></a> <span class="k">def</span> <span class="nf">_quote</span><span class="p">(</span><span class="n">expression</span><span class="p">:</span> <span class="n">E</span><span class="p">)</span> <span class="o">-></span> <span class="n">E</span><span class="p">:</span>
|
||||
</span><span id="quote_identifiers-423"><a href="#quote_identifiers-423"><span class="linenos">423</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">):</span>
|
||||
</span><span id="quote_identifiers-424"><a href="#quote_identifiers-424"><span class="linenos">424</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">expression</span><span class="o">.</span><span class="n">this</span>
|
||||
</span><span id="quote_identifiers-425"><a href="#quote_identifiers-425"><span class="linenos">425</span></a> <span class="n">expression</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="quote_identifiers-426"><a href="#quote_identifiers-426"><span class="linenos">426</span></a> <span class="s2">"quoted"</span><span class="p">,</span>
|
||||
</span><span id="quote_identifiers-427"><a href="#quote_identifiers-427"><span class="linenos">427</span></a> <span class="n">identify</span>
|
||||
</span><span id="quote_identifiers-428"><a href="#quote_identifiers-428"><span class="linenos">428</span></a> <span class="ow">or</span> <span class="n">case_sensitive</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="quote_identifiers-429"><a href="#quote_identifiers-429"><span class="linenos">429</span></a> <span class="ow">or</span> <span class="ow">not</span> <span class="n">exp</span><span class="o">.</span><span class="n">SAFE_IDENTIFIER_RE</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="n">name</span><span class="p">),</span>
|
||||
</span><span id="quote_identifiers-430"><a href="#quote_identifiers-430"><span class="linenos">430</span></a> <span class="p">)</span>
|
||||
</span><span id="quote_identifiers-431"><a href="#quote_identifiers-431"><span class="linenos">431</span></a> <span class="k">return</span> <span class="n">expression</span>
|
||||
</span><span id="quote_identifiers-432"><a href="#quote_identifiers-432"><span class="linenos">432</span></a>
|
||||
</span><span id="quote_identifiers-433"><a href="#quote_identifiers-433"><span class="linenos">433</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">_quote</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||||
</span><span id="quote_identifiers-421"><a href="#quote_identifiers-421"><span class="linenos">421</span></a> <span class="k">return</span> <span class="n">expression</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span>
|
||||
</span><span id="quote_identifiers-422"><a href="#quote_identifiers-422"><span class="linenos">422</span></a> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">quote_identifier</span><span class="p">,</span> <span class="n">identify</span><span class="o">=</span><span class="n">identify</span><span class="p">,</span> <span class="n">copy</span><span class="o">=</span><span class="kc">False</span>
|
||||
</span><span id="quote_identifiers-423"><a href="#quote_identifiers-423"><span class="linenos">423</span></a> <span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -825,139 +805,139 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#Resolver"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Resolver-436"><a href="#Resolver-436"><span class="linenos">436</span></a><span class="k">class</span> <span class="nc">Resolver</span><span class="p">:</span>
|
||||
</span><span id="Resolver-437"><a href="#Resolver-437"><span class="linenos">437</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="Resolver-438"><a href="#Resolver-438"><span class="linenos">438</span></a><span class="sd"> Helper for resolving columns.</span>
|
||||
</span><span id="Resolver-439"><a href="#Resolver-439"><span class="linenos">439</span></a>
|
||||
</span><span id="Resolver-440"><a href="#Resolver-440"><span class="linenos">440</span></a><span class="sd"> This is a class so we can lazily load some things and easily share them across functions.</span>
|
||||
</span><span id="Resolver-441"><a href="#Resolver-441"><span class="linenos">441</span></a><span class="sd"> """</span>
|
||||
</span><span id="Resolver-442"><a href="#Resolver-442"><span class="linenos">442</span></a>
|
||||
</span><span id="Resolver-443"><a href="#Resolver-443"><span class="linenos">443</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">infer_schema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">):</span>
|
||||
</span><span id="Resolver-444"><a href="#Resolver-444"><span class="linenos">444</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span> <span class="o">=</span> <span class="n">scope</span>
|
||||
</span><span id="Resolver-445"><a href="#Resolver-445"><span class="linenos">445</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span> <span class="o">=</span> <span class="n">schema</span>
|
||||
</span><span id="Resolver-446"><a href="#Resolver-446"><span class="linenos">446</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="Resolver-447"><a href="#Resolver-447"><span class="linenos">447</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="Resolver-448"><a href="#Resolver-448"><span class="linenos">448</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="Resolver-449"><a href="#Resolver-449"><span class="linenos">449</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_infer_schema</span> <span class="o">=</span> <span class="n">infer_schema</span>
|
||||
</span><span id="Resolver-450"><a href="#Resolver-450"><span class="linenos">450</span></a>
|
||||
</span><span id="Resolver-451"><a href="#Resolver-451"><span class="linenos">451</span></a> <span class="k">def</span> <span class="nf">get_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]:</span>
|
||||
</span><span id="Resolver-452"><a href="#Resolver-452"><span class="linenos">452</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="Resolver-453"><a href="#Resolver-453"><span class="linenos">453</span></a><span class="sd"> Get the table for a column name.</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Resolver-426"><a href="#Resolver-426"><span class="linenos">426</span></a><span class="k">class</span> <span class="nc">Resolver</span><span class="p">:</span>
|
||||
</span><span id="Resolver-427"><a href="#Resolver-427"><span class="linenos">427</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="Resolver-428"><a href="#Resolver-428"><span class="linenos">428</span></a><span class="sd"> Helper for resolving columns.</span>
|
||||
</span><span id="Resolver-429"><a href="#Resolver-429"><span class="linenos">429</span></a>
|
||||
</span><span id="Resolver-430"><a href="#Resolver-430"><span class="linenos">430</span></a><span class="sd"> This is a class so we can lazily load some things and easily share them across functions.</span>
|
||||
</span><span id="Resolver-431"><a href="#Resolver-431"><span class="linenos">431</span></a><span class="sd"> """</span>
|
||||
</span><span id="Resolver-432"><a href="#Resolver-432"><span class="linenos">432</span></a>
|
||||
</span><span id="Resolver-433"><a href="#Resolver-433"><span class="linenos">433</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">infer_schema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">):</span>
|
||||
</span><span id="Resolver-434"><a href="#Resolver-434"><span class="linenos">434</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span> <span class="o">=</span> <span class="n">scope</span>
|
||||
</span><span id="Resolver-435"><a href="#Resolver-435"><span class="linenos">435</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span> <span class="o">=</span> <span class="n">schema</span>
|
||||
</span><span id="Resolver-436"><a href="#Resolver-436"><span class="linenos">436</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="Resolver-437"><a href="#Resolver-437"><span class="linenos">437</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="Resolver-438"><a href="#Resolver-438"><span class="linenos">438</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="Resolver-439"><a href="#Resolver-439"><span class="linenos">439</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_infer_schema</span> <span class="o">=</span> <span class="n">infer_schema</span>
|
||||
</span><span id="Resolver-440"><a href="#Resolver-440"><span class="linenos">440</span></a>
|
||||
</span><span id="Resolver-441"><a href="#Resolver-441"><span class="linenos">441</span></a> <span class="k">def</span> <span class="nf">get_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]:</span>
|
||||
</span><span id="Resolver-442"><a href="#Resolver-442"><span class="linenos">442</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="Resolver-443"><a href="#Resolver-443"><span class="linenos">443</span></a><span class="sd"> Get the table for a column name.</span>
|
||||
</span><span id="Resolver-444"><a href="#Resolver-444"><span class="linenos">444</span></a>
|
||||
</span><span id="Resolver-445"><a href="#Resolver-445"><span class="linenos">445</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Resolver-446"><a href="#Resolver-446"><span class="linenos">446</span></a><span class="sd"> column_name: The column name to find the table for.</span>
|
||||
</span><span id="Resolver-447"><a href="#Resolver-447"><span class="linenos">447</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Resolver-448"><a href="#Resolver-448"><span class="linenos">448</span></a><span class="sd"> The table name if it can be found/inferred.</span>
|
||||
</span><span id="Resolver-449"><a href="#Resolver-449"><span class="linenos">449</span></a><span class="sd"> """</span>
|
||||
</span><span id="Resolver-450"><a href="#Resolver-450"><span class="linenos">450</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="Resolver-451"><a href="#Resolver-451"><span class="linenos">451</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_unambiguous_columns</span><span class="p">(</span>
|
||||
</span><span id="Resolver-452"><a href="#Resolver-452"><span class="linenos">452</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span>
|
||||
</span><span id="Resolver-453"><a href="#Resolver-453"><span class="linenos">453</span></a> <span class="p">)</span>
|
||||
</span><span id="Resolver-454"><a href="#Resolver-454"><span class="linenos">454</span></a>
|
||||
</span><span id="Resolver-455"><a href="#Resolver-455"><span class="linenos">455</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Resolver-456"><a href="#Resolver-456"><span class="linenos">456</span></a><span class="sd"> column_name: The column name to find the table for.</span>
|
||||
</span><span id="Resolver-457"><a href="#Resolver-457"><span class="linenos">457</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Resolver-458"><a href="#Resolver-458"><span class="linenos">458</span></a><span class="sd"> The table name if it can be found/inferred.</span>
|
||||
</span><span id="Resolver-459"><a href="#Resolver-459"><span class="linenos">459</span></a><span class="sd"> """</span>
|
||||
</span><span id="Resolver-460"><a href="#Resolver-460"><span class="linenos">460</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="Resolver-461"><a href="#Resolver-461"><span class="linenos">461</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_unambiguous_columns</span><span class="p">(</span>
|
||||
</span><span id="Resolver-462"><a href="#Resolver-462"><span class="linenos">462</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span>
|
||||
</span><span id="Resolver-463"><a href="#Resolver-463"><span class="linenos">463</span></a> <span class="p">)</span>
|
||||
</span><span id="Resolver-464"><a href="#Resolver-464"><span class="linenos">464</span></a>
|
||||
</span><span id="Resolver-465"><a href="#Resolver-465"><span class="linenos">465</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver-466"><a href="#Resolver-466"><span class="linenos">466</span></a>
|
||||
</span><span id="Resolver-467"><a href="#Resolver-467"><span class="linenos">467</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_name</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">_infer_schema</span><span class="p">:</span>
|
||||
</span><span id="Resolver-468"><a href="#Resolver-468"><span class="linenos">468</span></a> <span class="n">sources_without_schema</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span>
|
||||
</span><span id="Resolver-469"><a href="#Resolver-469"><span class="linenos">469</span></a> <span class="n">source</span>
|
||||
</span><span id="Resolver-470"><a href="#Resolver-470"><span class="linenos">470</span></a> <span class="k">for</span> <span class="n">source</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
||||
</span><span id="Resolver-471"><a href="#Resolver-471"><span class="linenos">471</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">columns</span> <span class="ow">or</span> <span class="s2">"*"</span> <span class="ow">in</span> <span class="n">columns</span>
|
||||
</span><span id="Resolver-472"><a href="#Resolver-472"><span class="linenos">472</span></a> <span class="p">)</span>
|
||||
</span><span id="Resolver-473"><a href="#Resolver-473"><span class="linenos">473</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">sources_without_schema</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="Resolver-474"><a href="#Resolver-474"><span class="linenos">474</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">sources_without_schema</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||
</span><span id="Resolver-475"><a href="#Resolver-475"><span class="linenos">475</span></a>
|
||||
</span><span id="Resolver-476"><a href="#Resolver-476"><span class="linenos">476</span></a> <span class="k">if</span> <span class="n">table_name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">:</span>
|
||||
</span><span id="Resolver-477"><a href="#Resolver-477"><span class="linenos">477</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver-455"><a href="#Resolver-455"><span class="linenos">455</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver-456"><a href="#Resolver-456"><span class="linenos">456</span></a>
|
||||
</span><span id="Resolver-457"><a href="#Resolver-457"><span class="linenos">457</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_name</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">_infer_schema</span><span class="p">:</span>
|
||||
</span><span id="Resolver-458"><a href="#Resolver-458"><span class="linenos">458</span></a> <span class="n">sources_without_schema</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span>
|
||||
</span><span id="Resolver-459"><a href="#Resolver-459"><span class="linenos">459</span></a> <span class="n">source</span>
|
||||
</span><span id="Resolver-460"><a href="#Resolver-460"><span class="linenos">460</span></a> <span class="k">for</span> <span class="n">source</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
||||
</span><span id="Resolver-461"><a href="#Resolver-461"><span class="linenos">461</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">columns</span> <span class="ow">or</span> <span class="s2">"*"</span> <span class="ow">in</span> <span class="n">columns</span>
|
||||
</span><span id="Resolver-462"><a href="#Resolver-462"><span class="linenos">462</span></a> <span class="p">)</span>
|
||||
</span><span id="Resolver-463"><a href="#Resolver-463"><span class="linenos">463</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">sources_without_schema</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="Resolver-464"><a href="#Resolver-464"><span class="linenos">464</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">sources_without_schema</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||
</span><span id="Resolver-465"><a href="#Resolver-465"><span class="linenos">465</span></a>
|
||||
</span><span id="Resolver-466"><a href="#Resolver-466"><span class="linenos">466</span></a> <span class="k">if</span> <span class="n">table_name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">:</span>
|
||||
</span><span id="Resolver-467"><a href="#Resolver-467"><span class="linenos">467</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver-468"><a href="#Resolver-468"><span class="linenos">468</span></a>
|
||||
</span><span id="Resolver-469"><a href="#Resolver-469"><span class="linenos">469</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">_</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver-470"><a href="#Resolver-470"><span class="linenos">470</span></a>
|
||||
</span><span id="Resolver-471"><a href="#Resolver-471"><span class="linenos">471</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subqueryable</span><span class="p">):</span>
|
||||
</span><span id="Resolver-472"><a href="#Resolver-472"><span class="linenos">472</span></a> <span class="k">while</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span> <span class="o">!=</span> <span class="n">table_name</span><span class="p">:</span>
|
||||
</span><span id="Resolver-473"><a href="#Resolver-473"><span class="linenos">473</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="Resolver-474"><a href="#Resolver-474"><span class="linenos">474</span></a>
|
||||
</span><span id="Resolver-475"><a href="#Resolver-475"><span class="linenos">475</span></a> <span class="n">node_alias</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span>
|
||||
</span><span id="Resolver-476"><a href="#Resolver-476"><span class="linenos">476</span></a> <span class="k">if</span> <span class="n">node_alias</span><span class="p">:</span>
|
||||
</span><span id="Resolver-477"><a href="#Resolver-477"><span class="linenos">477</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">node_alias</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="Resolver-478"><a href="#Resolver-478"><span class="linenos">478</span></a>
|
||||
</span><span id="Resolver-479"><a href="#Resolver-479"><span class="linenos">479</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">_</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver-479"><a href="#Resolver-479"><span class="linenos">479</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver-480"><a href="#Resolver-480"><span class="linenos">480</span></a>
|
||||
</span><span id="Resolver-481"><a href="#Resolver-481"><span class="linenos">481</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subqueryable</span><span class="p">):</span>
|
||||
</span><span id="Resolver-482"><a href="#Resolver-482"><span class="linenos">482</span></a> <span class="k">while</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span> <span class="o">!=</span> <span class="n">table_name</span><span class="p">:</span>
|
||||
</span><span id="Resolver-483"><a href="#Resolver-483"><span class="linenos">483</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="Resolver-484"><a href="#Resolver-484"><span class="linenos">484</span></a>
|
||||
</span><span id="Resolver-485"><a href="#Resolver-485"><span class="linenos">485</span></a> <span class="n">node_alias</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span>
|
||||
</span><span id="Resolver-486"><a href="#Resolver-486"><span class="linenos">486</span></a> <span class="k">if</span> <span class="n">node_alias</span><span class="p">:</span>
|
||||
</span><span id="Resolver-487"><a href="#Resolver-487"><span class="linenos">487</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">node_alias</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="Resolver-488"><a href="#Resolver-488"><span class="linenos">488</span></a>
|
||||
</span><span id="Resolver-489"><a href="#Resolver-489"><span class="linenos">489</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver-490"><a href="#Resolver-490"><span class="linenos">490</span></a>
|
||||
</span><span id="Resolver-491"><a href="#Resolver-491"><span class="linenos">491</span></a> <span class="nd">@property</span>
|
||||
</span><span id="Resolver-492"><a href="#Resolver-492"><span class="linenos">492</span></a> <span class="k">def</span> <span class="nf">all_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Resolver-493"><a href="#Resolver-493"><span class="linenos">493</span></a><span class="w"> </span><span class="sd">"""All available columns of all sources in this scope"""</span>
|
||||
</span><span id="Resolver-494"><a href="#Resolver-494"><span class="linenos">494</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="Resolver-495"><a href="#Resolver-495"><span class="linenos">495</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="Resolver-496"><a href="#Resolver-496"><span class="linenos">496</span></a> <span class="n">column</span> <span class="k">for</span> <span class="n">columns</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span><span class="o">.</span><span class="n">values</span><span class="p">()</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span>
|
||||
</span><span id="Resolver-497"><a href="#Resolver-497"><span class="linenos">497</span></a> <span class="p">}</span>
|
||||
</span><span id="Resolver-498"><a href="#Resolver-498"><span class="linenos">498</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span>
|
||||
</span><span id="Resolver-499"><a href="#Resolver-499"><span class="linenos">499</span></a>
|
||||
</span><span id="Resolver-500"><a href="#Resolver-500"><span class="linenos">500</span></a> <span class="k">def</span> <span class="nf">get_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">only_visible</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
|
||||
</span><span id="Resolver-501"><a href="#Resolver-501"><span class="linenos">501</span></a><span class="w"> </span><span class="sd">"""Resolve the source columns for a given source `name`"""</span>
|
||||
</span><span id="Resolver-502"><a href="#Resolver-502"><span class="linenos">502</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="p">:</span>
|
||||
</span><span id="Resolver-503"><a href="#Resolver-503"><span class="linenos">503</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown table: </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="Resolver-504"><a href="#Resolver-504"><span class="linenos">504</span></a>
|
||||
</span><span id="Resolver-505"><a href="#Resolver-505"><span class="linenos">505</span></a> <span class="n">source</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
|
||||
</span><span id="Resolver-481"><a href="#Resolver-481"><span class="linenos">481</span></a> <span class="nd">@property</span>
|
||||
</span><span id="Resolver-482"><a href="#Resolver-482"><span class="linenos">482</span></a> <span class="k">def</span> <span class="nf">all_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Resolver-483"><a href="#Resolver-483"><span class="linenos">483</span></a><span class="w"> </span><span class="sd">"""All available columns of all sources in this scope"""</span>
|
||||
</span><span id="Resolver-484"><a href="#Resolver-484"><span class="linenos">484</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="Resolver-485"><a href="#Resolver-485"><span class="linenos">485</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="Resolver-486"><a href="#Resolver-486"><span class="linenos">486</span></a> <span class="n">column</span> <span class="k">for</span> <span class="n">columns</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span><span class="o">.</span><span class="n">values</span><span class="p">()</span> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span>
|
||||
</span><span id="Resolver-487"><a href="#Resolver-487"><span class="linenos">487</span></a> <span class="p">}</span>
|
||||
</span><span id="Resolver-488"><a href="#Resolver-488"><span class="linenos">488</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span>
|
||||
</span><span id="Resolver-489"><a href="#Resolver-489"><span class="linenos">489</span></a>
|
||||
</span><span id="Resolver-490"><a href="#Resolver-490"><span class="linenos">490</span></a> <span class="k">def</span> <span class="nf">get_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">only_visible</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
|
||||
</span><span id="Resolver-491"><a href="#Resolver-491"><span class="linenos">491</span></a><span class="w"> </span><span class="sd">"""Resolve the source columns for a given source `name`"""</span>
|
||||
</span><span id="Resolver-492"><a href="#Resolver-492"><span class="linenos">492</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="p">:</span>
|
||||
</span><span id="Resolver-493"><a href="#Resolver-493"><span class="linenos">493</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown table: </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="Resolver-494"><a href="#Resolver-494"><span class="linenos">494</span></a>
|
||||
</span><span id="Resolver-495"><a href="#Resolver-495"><span class="linenos">495</span></a> <span class="n">source</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
|
||||
</span><span id="Resolver-496"><a href="#Resolver-496"><span class="linenos">496</span></a>
|
||||
</span><span id="Resolver-497"><a href="#Resolver-497"><span class="linenos">497</span></a> <span class="c1"># If referencing a table, return the columns from the schema</span>
|
||||
</span><span id="Resolver-498"><a href="#Resolver-498"><span class="linenos">498</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="Resolver-499"><a href="#Resolver-499"><span class="linenos">499</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span><span class="o">.</span><span class="n">column_names</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">only_visible</span><span class="p">)</span>
|
||||
</span><span id="Resolver-500"><a href="#Resolver-500"><span class="linenos">500</span></a>
|
||||
</span><span id="Resolver-501"><a href="#Resolver-501"><span class="linenos">501</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">):</span>
|
||||
</span><span id="Resolver-502"><a href="#Resolver-502"><span class="linenos">502</span></a> <span class="k">return</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">alias_column_names</span>
|
||||
</span><span id="Resolver-503"><a href="#Resolver-503"><span class="linenos">503</span></a>
|
||||
</span><span id="Resolver-504"><a href="#Resolver-504"><span class="linenos">504</span></a> <span class="c1"># Otherwise, if referencing another scope, return that scope's named selects</span>
|
||||
</span><span id="Resolver-505"><a href="#Resolver-505"><span class="linenos">505</span></a> <span class="k">return</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span>
|
||||
</span><span id="Resolver-506"><a href="#Resolver-506"><span class="linenos">506</span></a>
|
||||
</span><span id="Resolver-507"><a href="#Resolver-507"><span class="linenos">507</span></a> <span class="c1"># If referencing a table, return the columns from the schema</span>
|
||||
</span><span id="Resolver-508"><a href="#Resolver-508"><span class="linenos">508</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="Resolver-509"><a href="#Resolver-509"><span class="linenos">509</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span><span class="o">.</span><span class="n">column_names</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">only_visible</span><span class="p">)</span>
|
||||
</span><span id="Resolver-510"><a href="#Resolver-510"><span class="linenos">510</span></a>
|
||||
</span><span id="Resolver-511"><a href="#Resolver-511"><span class="linenos">511</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">):</span>
|
||||
</span><span id="Resolver-512"><a href="#Resolver-512"><span class="linenos">512</span></a> <span class="k">return</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">alias_column_names</span>
|
||||
</span><span id="Resolver-513"><a href="#Resolver-513"><span class="linenos">513</span></a>
|
||||
</span><span id="Resolver-514"><a href="#Resolver-514"><span class="linenos">514</span></a> <span class="c1"># Otherwise, if referencing another scope, return that scope's named selects</span>
|
||||
</span><span id="Resolver-515"><a href="#Resolver-515"><span class="linenos">515</span></a> <span class="k">return</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span>
|
||||
</span><span id="Resolver-516"><a href="#Resolver-516"><span class="linenos">516</span></a>
|
||||
</span><span id="Resolver-517"><a href="#Resolver-517"><span class="linenos">517</span></a> <span class="k">def</span> <span class="nf">_get_all_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Resolver-518"><a href="#Resolver-518"><span class="linenos">518</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="Resolver-519"><a href="#Resolver-519"><span class="linenos">519</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="Resolver-520"><a href="#Resolver-520"><span class="linenos">520</span></a> <span class="n">k</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_source_columns</span><span class="p">(</span><span class="n">k</span><span class="p">)</span>
|
||||
</span><span id="Resolver-521"><a href="#Resolver-521"><span class="linenos">521</span></a> <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">lateral_sources</span><span class="p">)</span>
|
||||
</span><span id="Resolver-522"><a href="#Resolver-522"><span class="linenos">522</span></a> <span class="p">}</span>
|
||||
</span><span id="Resolver-523"><a href="#Resolver-523"><span class="linenos">523</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span>
|
||||
</span><span id="Resolver-524"><a href="#Resolver-524"><span class="linenos">524</span></a>
|
||||
</span><span id="Resolver-525"><a href="#Resolver-525"><span class="linenos">525</span></a> <span class="k">def</span> <span class="nf">_get_unambiguous_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">source_columns</span><span class="p">):</span>
|
||||
</span><span id="Resolver-526"><a href="#Resolver-526"><span class="linenos">526</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="Resolver-527"><a href="#Resolver-527"><span class="linenos">527</span></a><span class="sd"> Find all the unambiguous columns in sources.</span>
|
||||
</span><span id="Resolver-507"><a href="#Resolver-507"><span class="linenos">507</span></a> <span class="k">def</span> <span class="nf">_get_all_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
|
||||
</span><span id="Resolver-508"><a href="#Resolver-508"><span class="linenos">508</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="Resolver-509"><a href="#Resolver-509"><span class="linenos">509</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="Resolver-510"><a href="#Resolver-510"><span class="linenos">510</span></a> <span class="n">k</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_source_columns</span><span class="p">(</span><span class="n">k</span><span class="p">)</span>
|
||||
</span><span id="Resolver-511"><a href="#Resolver-511"><span class="linenos">511</span></a> <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">itertools</span><span class="o">.</span><span class="n">chain</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">lateral_sources</span><span class="p">)</span>
|
||||
</span><span id="Resolver-512"><a href="#Resolver-512"><span class="linenos">512</span></a> <span class="p">}</span>
|
||||
</span><span id="Resolver-513"><a href="#Resolver-513"><span class="linenos">513</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span>
|
||||
</span><span id="Resolver-514"><a href="#Resolver-514"><span class="linenos">514</span></a>
|
||||
</span><span id="Resolver-515"><a href="#Resolver-515"><span class="linenos">515</span></a> <span class="k">def</span> <span class="nf">_get_unambiguous_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">source_columns</span><span class="p">):</span>
|
||||
</span><span id="Resolver-516"><a href="#Resolver-516"><span class="linenos">516</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="Resolver-517"><a href="#Resolver-517"><span class="linenos">517</span></a><span class="sd"> Find all the unambiguous columns in sources.</span>
|
||||
</span><span id="Resolver-518"><a href="#Resolver-518"><span class="linenos">518</span></a>
|
||||
</span><span id="Resolver-519"><a href="#Resolver-519"><span class="linenos">519</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Resolver-520"><a href="#Resolver-520"><span class="linenos">520</span></a><span class="sd"> source_columns (dict): Mapping of names to source columns</span>
|
||||
</span><span id="Resolver-521"><a href="#Resolver-521"><span class="linenos">521</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Resolver-522"><a href="#Resolver-522"><span class="linenos">522</span></a><span class="sd"> dict: Mapping of column name to source name</span>
|
||||
</span><span id="Resolver-523"><a href="#Resolver-523"><span class="linenos">523</span></a><span class="sd"> """</span>
|
||||
</span><span id="Resolver-524"><a href="#Resolver-524"><span class="linenos">524</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source_columns</span><span class="p">:</span>
|
||||
</span><span id="Resolver-525"><a href="#Resolver-525"><span class="linenos">525</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="Resolver-526"><a href="#Resolver-526"><span class="linenos">526</span></a>
|
||||
</span><span id="Resolver-527"><a href="#Resolver-527"><span class="linenos">527</span></a> <span class="n">source_columns</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">source_columns</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
|
||||
</span><span id="Resolver-528"><a href="#Resolver-528"><span class="linenos">528</span></a>
|
||||
</span><span id="Resolver-529"><a href="#Resolver-529"><span class="linenos">529</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Resolver-530"><a href="#Resolver-530"><span class="linenos">530</span></a><span class="sd"> source_columns (dict): Mapping of names to source columns</span>
|
||||
</span><span id="Resolver-531"><a href="#Resolver-531"><span class="linenos">531</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Resolver-532"><a href="#Resolver-532"><span class="linenos">532</span></a><span class="sd"> dict: Mapping of column name to source name</span>
|
||||
</span><span id="Resolver-533"><a href="#Resolver-533"><span class="linenos">533</span></a><span class="sd"> """</span>
|
||||
</span><span id="Resolver-534"><a href="#Resolver-534"><span class="linenos">534</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">source_columns</span><span class="p">:</span>
|
||||
</span><span id="Resolver-535"><a href="#Resolver-535"><span class="linenos">535</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="Resolver-536"><a href="#Resolver-536"><span class="linenos">536</span></a>
|
||||
</span><span id="Resolver-537"><a href="#Resolver-537"><span class="linenos">537</span></a> <span class="n">source_columns</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">source_columns</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
|
||||
</span><span id="Resolver-538"><a href="#Resolver-538"><span class="linenos">538</span></a>
|
||||
</span><span id="Resolver-539"><a href="#Resolver-539"><span class="linenos">539</span></a> <span class="n">first_table</span><span class="p">,</span> <span class="n">first_columns</span> <span class="o">=</span> <span class="n">source_columns</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||
</span><span id="Resolver-540"><a href="#Resolver-540"><span class="linenos">540</span></a> <span class="n">unambiguous_columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">col</span><span class="p">:</span> <span class="n">first_table</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">first_columns</span><span class="p">)}</span>
|
||||
</span><span id="Resolver-541"><a href="#Resolver-541"><span class="linenos">541</span></a> <span class="n">all_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">unambiguous_columns</span><span class="p">)</span>
|
||||
</span><span id="Resolver-542"><a href="#Resolver-542"><span class="linenos">542</span></a>
|
||||
</span><span id="Resolver-543"><a href="#Resolver-543"><span class="linenos">543</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="n">source_columns</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
||||
</span><span id="Resolver-544"><a href="#Resolver-544"><span class="linenos">544</span></a> <span class="n">unique</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Resolver-545"><a href="#Resolver-545"><span class="linenos">545</span></a> <span class="n">ambiguous</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_columns</span><span class="p">)</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="n">unique</span><span class="p">)</span>
|
||||
</span><span id="Resolver-546"><a href="#Resolver-546"><span class="linenos">546</span></a> <span class="n">all_columns</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Resolver-547"><a href="#Resolver-547"><span class="linenos">547</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">ambiguous</span><span class="p">:</span>
|
||||
</span><span id="Resolver-548"><a href="#Resolver-548"><span class="linenos">548</span></a> <span class="n">unambiguous_columns</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="Resolver-549"><a href="#Resolver-549"><span class="linenos">549</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">unique</span><span class="o">.</span><span class="n">difference</span><span class="p">(</span><span class="n">ambiguous</span><span class="p">):</span>
|
||||
</span><span id="Resolver-550"><a href="#Resolver-550"><span class="linenos">550</span></a> <span class="n">unambiguous_columns</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="Resolver-551"><a href="#Resolver-551"><span class="linenos">551</span></a>
|
||||
</span><span id="Resolver-552"><a href="#Resolver-552"><span class="linenos">552</span></a> <span class="k">return</span> <span class="n">unambiguous_columns</span>
|
||||
</span><span id="Resolver-553"><a href="#Resolver-553"><span class="linenos">553</span></a>
|
||||
</span><span id="Resolver-554"><a href="#Resolver-554"><span class="linenos">554</span></a> <span class="nd">@staticmethod</span>
|
||||
</span><span id="Resolver-555"><a href="#Resolver-555"><span class="linenos">555</span></a> <span class="k">def</span> <span class="nf">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">):</span>
|
||||
</span><span id="Resolver-556"><a href="#Resolver-556"><span class="linenos">556</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="Resolver-557"><a href="#Resolver-557"><span class="linenos">557</span></a><span class="sd"> Find the unique columns in a list of columns.</span>
|
||||
</span><span id="Resolver-558"><a href="#Resolver-558"><span class="linenos">558</span></a>
|
||||
</span><span id="Resolver-559"><a href="#Resolver-559"><span class="linenos">559</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="Resolver-560"><a href="#Resolver-560"><span class="linenos">560</span></a><span class="sd"> >>> sorted(Resolver._find_unique_columns(["a", "b", "b", "c"]))</span>
|
||||
</span><span id="Resolver-561"><a href="#Resolver-561"><span class="linenos">561</span></a><span class="sd"> ['a', 'c']</span>
|
||||
</span><span id="Resolver-562"><a href="#Resolver-562"><span class="linenos">562</span></a>
|
||||
</span><span id="Resolver-563"><a href="#Resolver-563"><span class="linenos">563</span></a><span class="sd"> This is necessary because duplicate column names are ambiguous.</span>
|
||||
</span><span id="Resolver-564"><a href="#Resolver-564"><span class="linenos">564</span></a><span class="sd"> """</span>
|
||||
</span><span id="Resolver-565"><a href="#Resolver-565"><span class="linenos">565</span></a> <span class="n">counts</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="Resolver-566"><a href="#Resolver-566"><span class="linenos">566</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="Resolver-567"><a href="#Resolver-567"><span class="linenos">567</span></a> <span class="n">counts</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">counts</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
|
||||
</span><span id="Resolver-568"><a href="#Resolver-568"><span class="linenos">568</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span><span class="p">,</span> <span class="n">count</span> <span class="ow">in</span> <span class="n">counts</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">count</span> <span class="o">==</span> <span class="mi">1</span><span class="p">}</span>
|
||||
</span><span id="Resolver-529"><a href="#Resolver-529"><span class="linenos">529</span></a> <span class="n">first_table</span><span class="p">,</span> <span class="n">first_columns</span> <span class="o">=</span> <span class="n">source_columns</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||
</span><span id="Resolver-530"><a href="#Resolver-530"><span class="linenos">530</span></a> <span class="n">unambiguous_columns</span> <span class="o">=</span> <span class="p">{</span><span class="n">col</span><span class="p">:</span> <span class="n">first_table</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">first_columns</span><span class="p">)}</span>
|
||||
</span><span id="Resolver-531"><a href="#Resolver-531"><span class="linenos">531</span></a> <span class="n">all_columns</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">unambiguous_columns</span><span class="p">)</span>
|
||||
</span><span id="Resolver-532"><a href="#Resolver-532"><span class="linenos">532</span></a>
|
||||
</span><span id="Resolver-533"><a href="#Resolver-533"><span class="linenos">533</span></a> <span class="k">for</span> <span class="n">table</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="n">source_columns</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
|
||||
</span><span id="Resolver-534"><a href="#Resolver-534"><span class="linenos">534</span></a> <span class="n">unique</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Resolver-535"><a href="#Resolver-535"><span class="linenos">535</span></a> <span class="n">ambiguous</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_columns</span><span class="p">)</span><span class="o">.</span><span class="n">intersection</span><span class="p">(</span><span class="n">unique</span><span class="p">)</span>
|
||||
</span><span id="Resolver-536"><a href="#Resolver-536"><span class="linenos">536</span></a> <span class="n">all_columns</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">columns</span><span class="p">)</span>
|
||||
</span><span id="Resolver-537"><a href="#Resolver-537"><span class="linenos">537</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">ambiguous</span><span class="p">:</span>
|
||||
</span><span id="Resolver-538"><a href="#Resolver-538"><span class="linenos">538</span></a> <span class="n">unambiguous_columns</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
|
||||
</span><span id="Resolver-539"><a href="#Resolver-539"><span class="linenos">539</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">unique</span><span class="o">.</span><span class="n">difference</span><span class="p">(</span><span class="n">ambiguous</span><span class="p">):</span>
|
||||
</span><span id="Resolver-540"><a href="#Resolver-540"><span class="linenos">540</span></a> <span class="n">unambiguous_columns</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">table</span>
|
||||
</span><span id="Resolver-541"><a href="#Resolver-541"><span class="linenos">541</span></a>
|
||||
</span><span id="Resolver-542"><a href="#Resolver-542"><span class="linenos">542</span></a> <span class="k">return</span> <span class="n">unambiguous_columns</span>
|
||||
</span><span id="Resolver-543"><a href="#Resolver-543"><span class="linenos">543</span></a>
|
||||
</span><span id="Resolver-544"><a href="#Resolver-544"><span class="linenos">544</span></a> <span class="nd">@staticmethod</span>
|
||||
</span><span id="Resolver-545"><a href="#Resolver-545"><span class="linenos">545</span></a> <span class="k">def</span> <span class="nf">_find_unique_columns</span><span class="p">(</span><span class="n">columns</span><span class="p">):</span>
|
||||
</span><span id="Resolver-546"><a href="#Resolver-546"><span class="linenos">546</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="Resolver-547"><a href="#Resolver-547"><span class="linenos">547</span></a><span class="sd"> Find the unique columns in a list of columns.</span>
|
||||
</span><span id="Resolver-548"><a href="#Resolver-548"><span class="linenos">548</span></a>
|
||||
</span><span id="Resolver-549"><a href="#Resolver-549"><span class="linenos">549</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="Resolver-550"><a href="#Resolver-550"><span class="linenos">550</span></a><span class="sd"> >>> sorted(Resolver._find_unique_columns(["a", "b", "b", "c"]))</span>
|
||||
</span><span id="Resolver-551"><a href="#Resolver-551"><span class="linenos">551</span></a><span class="sd"> ['a', 'c']</span>
|
||||
</span><span id="Resolver-552"><a href="#Resolver-552"><span class="linenos">552</span></a>
|
||||
</span><span id="Resolver-553"><a href="#Resolver-553"><span class="linenos">553</span></a><span class="sd"> This is necessary because duplicate column names are ambiguous.</span>
|
||||
</span><span id="Resolver-554"><a href="#Resolver-554"><span class="linenos">554</span></a><span class="sd"> """</span>
|
||||
</span><span id="Resolver-555"><a href="#Resolver-555"><span class="linenos">555</span></a> <span class="n">counts</span> <span class="o">=</span> <span class="p">{}</span>
|
||||
</span><span id="Resolver-556"><a href="#Resolver-556"><span class="linenos">556</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">columns</span><span class="p">:</span>
|
||||
</span><span id="Resolver-557"><a href="#Resolver-557"><span class="linenos">557</span></a> <span class="n">counts</span><span class="p">[</span><span class="n">column</span><span class="p">]</span> <span class="o">=</span> <span class="n">counts</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span>
|
||||
</span><span id="Resolver-558"><a href="#Resolver-558"><span class="linenos">558</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">column</span> <span class="k">for</span> <span class="n">column</span><span class="p">,</span> <span class="n">count</span> <span class="ow">in</span> <span class="n">counts</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">count</span> <span class="o">==</span> <span class="mi">1</span><span class="p">}</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -977,13 +957,13 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#Resolver.__init__"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Resolver.__init__-443"><a href="#Resolver.__init__-443"><span class="linenos">443</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">infer_schema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">):</span>
|
||||
</span><span id="Resolver.__init__-444"><a href="#Resolver.__init__-444"><span class="linenos">444</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span> <span class="o">=</span> <span class="n">scope</span>
|
||||
</span><span id="Resolver.__init__-445"><a href="#Resolver.__init__-445"><span class="linenos">445</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span> <span class="o">=</span> <span class="n">schema</span>
|
||||
</span><span id="Resolver.__init__-446"><a href="#Resolver.__init__-446"><span class="linenos">446</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="Resolver.__init__-447"><a href="#Resolver.__init__-447"><span class="linenos">447</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="Resolver.__init__-448"><a href="#Resolver.__init__-448"><span class="linenos">448</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="Resolver.__init__-449"><a href="#Resolver.__init__-449"><span class="linenos">449</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_infer_schema</span> <span class="o">=</span> <span class="n">infer_schema</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Resolver.__init__-433"><a href="#Resolver.__init__-433"><span class="linenos">433</span></a> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">scope</span><span class="p">,</span> <span class="n">schema</span><span class="p">,</span> <span class="n">infer_schema</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span><span class="p">):</span>
|
||||
</span><span id="Resolver.__init__-434"><a href="#Resolver.__init__-434"><span class="linenos">434</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span> <span class="o">=</span> <span class="n">scope</span>
|
||||
</span><span id="Resolver.__init__-435"><a href="#Resolver.__init__-435"><span class="linenos">435</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span> <span class="o">=</span> <span class="n">schema</span>
|
||||
</span><span id="Resolver.__init__-436"><a href="#Resolver.__init__-436"><span class="linenos">436</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_source_columns</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="Resolver.__init__-437"><a href="#Resolver.__init__-437"><span class="linenos">437</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="Resolver.__init__-438"><a href="#Resolver.__init__-438"><span class="linenos">438</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_all_columns</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="Resolver.__init__-439"><a href="#Resolver.__init__-439"><span class="linenos">439</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_infer_schema</span> <span class="o">=</span> <span class="n">infer_schema</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1001,45 +981,45 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#Resolver.get_table"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Resolver.get_table-451"><a href="#Resolver.get_table-451"><span class="linenos">451</span></a> <span class="k">def</span> <span class="nf">get_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]:</span>
|
||||
</span><span id="Resolver.get_table-452"><a href="#Resolver.get_table-452"><span class="linenos">452</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="Resolver.get_table-453"><a href="#Resolver.get_table-453"><span class="linenos">453</span></a><span class="sd"> Get the table for a column name.</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Resolver.get_table-441"><a href="#Resolver.get_table-441"><span class="linenos">441</span></a> <span class="k">def</span> <span class="nf">get_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">column_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">]:</span>
|
||||
</span><span id="Resolver.get_table-442"><a href="#Resolver.get_table-442"><span class="linenos">442</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="Resolver.get_table-443"><a href="#Resolver.get_table-443"><span class="linenos">443</span></a><span class="sd"> Get the table for a column name.</span>
|
||||
</span><span id="Resolver.get_table-444"><a href="#Resolver.get_table-444"><span class="linenos">444</span></a>
|
||||
</span><span id="Resolver.get_table-445"><a href="#Resolver.get_table-445"><span class="linenos">445</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Resolver.get_table-446"><a href="#Resolver.get_table-446"><span class="linenos">446</span></a><span class="sd"> column_name: The column name to find the table for.</span>
|
||||
</span><span id="Resolver.get_table-447"><a href="#Resolver.get_table-447"><span class="linenos">447</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Resolver.get_table-448"><a href="#Resolver.get_table-448"><span class="linenos">448</span></a><span class="sd"> The table name if it can be found/inferred.</span>
|
||||
</span><span id="Resolver.get_table-449"><a href="#Resolver.get_table-449"><span class="linenos">449</span></a><span class="sd"> """</span>
|
||||
</span><span id="Resolver.get_table-450"><a href="#Resolver.get_table-450"><span class="linenos">450</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="Resolver.get_table-451"><a href="#Resolver.get_table-451"><span class="linenos">451</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_unambiguous_columns</span><span class="p">(</span>
|
||||
</span><span id="Resolver.get_table-452"><a href="#Resolver.get_table-452"><span class="linenos">452</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span>
|
||||
</span><span id="Resolver.get_table-453"><a href="#Resolver.get_table-453"><span class="linenos">453</span></a> <span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-454"><a href="#Resolver.get_table-454"><span class="linenos">454</span></a>
|
||||
</span><span id="Resolver.get_table-455"><a href="#Resolver.get_table-455"><span class="linenos">455</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="Resolver.get_table-456"><a href="#Resolver.get_table-456"><span class="linenos">456</span></a><span class="sd"> column_name: The column name to find the table for.</span>
|
||||
</span><span id="Resolver.get_table-457"><a href="#Resolver.get_table-457"><span class="linenos">457</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="Resolver.get_table-458"><a href="#Resolver.get_table-458"><span class="linenos">458</span></a><span class="sd"> The table name if it can be found/inferred.</span>
|
||||
</span><span id="Resolver.get_table-459"><a href="#Resolver.get_table-459"><span class="linenos">459</span></a><span class="sd"> """</span>
|
||||
</span><span id="Resolver.get_table-460"><a href="#Resolver.get_table-460"><span class="linenos">460</span></a> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="Resolver.get_table-461"><a href="#Resolver.get_table-461"><span class="linenos">461</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_unambiguous_columns</span><span class="p">(</span>
|
||||
</span><span id="Resolver.get_table-462"><a href="#Resolver.get_table-462"><span class="linenos">462</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span>
|
||||
</span><span id="Resolver.get_table-463"><a href="#Resolver.get_table-463"><span class="linenos">463</span></a> <span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-464"><a href="#Resolver.get_table-464"><span class="linenos">464</span></a>
|
||||
</span><span id="Resolver.get_table-465"><a href="#Resolver.get_table-465"><span class="linenos">465</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-466"><a href="#Resolver.get_table-466"><span class="linenos">466</span></a>
|
||||
</span><span id="Resolver.get_table-467"><a href="#Resolver.get_table-467"><span class="linenos">467</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_name</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">_infer_schema</span><span class="p">:</span>
|
||||
</span><span id="Resolver.get_table-468"><a href="#Resolver.get_table-468"><span class="linenos">468</span></a> <span class="n">sources_without_schema</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span>
|
||||
</span><span id="Resolver.get_table-469"><a href="#Resolver.get_table-469"><span class="linenos">469</span></a> <span class="n">source</span>
|
||||
</span><span id="Resolver.get_table-470"><a href="#Resolver.get_table-470"><span class="linenos">470</span></a> <span class="k">for</span> <span class="n">source</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
||||
</span><span id="Resolver.get_table-471"><a href="#Resolver.get_table-471"><span class="linenos">471</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">columns</span> <span class="ow">or</span> <span class="s2">"*"</span> <span class="ow">in</span> <span class="n">columns</span>
|
||||
</span><span id="Resolver.get_table-472"><a href="#Resolver.get_table-472"><span class="linenos">472</span></a> <span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-473"><a href="#Resolver.get_table-473"><span class="linenos">473</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">sources_without_schema</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="Resolver.get_table-474"><a href="#Resolver.get_table-474"><span class="linenos">474</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">sources_without_schema</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||
</span><span id="Resolver.get_table-475"><a href="#Resolver.get_table-475"><span class="linenos">475</span></a>
|
||||
</span><span id="Resolver.get_table-476"><a href="#Resolver.get_table-476"><span class="linenos">476</span></a> <span class="k">if</span> <span class="n">table_name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">:</span>
|
||||
</span><span id="Resolver.get_table-477"><a href="#Resolver.get_table-477"><span class="linenos">477</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-455"><a href="#Resolver.get_table-455"><span class="linenos">455</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_unambiguous_columns</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">column_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-456"><a href="#Resolver.get_table-456"><span class="linenos">456</span></a>
|
||||
</span><span id="Resolver.get_table-457"><a href="#Resolver.get_table-457"><span class="linenos">457</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">table_name</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">_infer_schema</span><span class="p">:</span>
|
||||
</span><span id="Resolver.get_table-458"><a href="#Resolver.get_table-458"><span class="linenos">458</span></a> <span class="n">sources_without_schema</span> <span class="o">=</span> <span class="nb">tuple</span><span class="p">(</span>
|
||||
</span><span id="Resolver.get_table-459"><a href="#Resolver.get_table-459"><span class="linenos">459</span></a> <span class="n">source</span>
|
||||
</span><span id="Resolver.get_table-460"><a href="#Resolver.get_table-460"><span class="linenos">460</span></a> <span class="k">for</span> <span class="n">source</span><span class="p">,</span> <span class="n">columns</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_get_all_source_columns</span><span class="p">()</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
|
||||
</span><span id="Resolver.get_table-461"><a href="#Resolver.get_table-461"><span class="linenos">461</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">columns</span> <span class="ow">or</span> <span class="s2">"*"</span> <span class="ow">in</span> <span class="n">columns</span>
|
||||
</span><span id="Resolver.get_table-462"><a href="#Resolver.get_table-462"><span class="linenos">462</span></a> <span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-463"><a href="#Resolver.get_table-463"><span class="linenos">463</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">sources_without_schema</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="Resolver.get_table-464"><a href="#Resolver.get_table-464"><span class="linenos">464</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">sources_without_schema</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||||
</span><span id="Resolver.get_table-465"><a href="#Resolver.get_table-465"><span class="linenos">465</span></a>
|
||||
</span><span id="Resolver.get_table-466"><a href="#Resolver.get_table-466"><span class="linenos">466</span></a> <span class="k">if</span> <span class="n">table_name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="p">:</span>
|
||||
</span><span id="Resolver.get_table-467"><a href="#Resolver.get_table-467"><span class="linenos">467</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-468"><a href="#Resolver.get_table-468"><span class="linenos">468</span></a>
|
||||
</span><span id="Resolver.get_table-469"><a href="#Resolver.get_table-469"><span class="linenos">469</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">_</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-470"><a href="#Resolver.get_table-470"><span class="linenos">470</span></a>
|
||||
</span><span id="Resolver.get_table-471"><a href="#Resolver.get_table-471"><span class="linenos">471</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subqueryable</span><span class="p">):</span>
|
||||
</span><span id="Resolver.get_table-472"><a href="#Resolver.get_table-472"><span class="linenos">472</span></a> <span class="k">while</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span> <span class="o">!=</span> <span class="n">table_name</span><span class="p">:</span>
|
||||
</span><span id="Resolver.get_table-473"><a href="#Resolver.get_table-473"><span class="linenos">473</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="Resolver.get_table-474"><a href="#Resolver.get_table-474"><span class="linenos">474</span></a>
|
||||
</span><span id="Resolver.get_table-475"><a href="#Resolver.get_table-475"><span class="linenos">475</span></a> <span class="n">node_alias</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-476"><a href="#Resolver.get_table-476"><span class="linenos">476</span></a> <span class="k">if</span> <span class="n">node_alias</span><span class="p">:</span>
|
||||
</span><span id="Resolver.get_table-477"><a href="#Resolver.get_table-477"><span class="linenos">477</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">node_alias</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-478"><a href="#Resolver.get_table-478"><span class="linenos">478</span></a>
|
||||
</span><span id="Resolver.get_table-479"><a href="#Resolver.get_table-479"><span class="linenos">479</span></a> <span class="n">node</span><span class="p">,</span> <span class="n">_</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">selected_sources</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-480"><a href="#Resolver.get_table-480"><span class="linenos">480</span></a>
|
||||
</span><span id="Resolver.get_table-481"><a href="#Resolver.get_table-481"><span class="linenos">481</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Subqueryable</span><span class="p">):</span>
|
||||
</span><span id="Resolver.get_table-482"><a href="#Resolver.get_table-482"><span class="linenos">482</span></a> <span class="k">while</span> <span class="n">node</span> <span class="ow">and</span> <span class="n">node</span><span class="o">.</span><span class="n">alias</span> <span class="o">!=</span> <span class="n">table_name</span><span class="p">:</span>
|
||||
</span><span id="Resolver.get_table-483"><a href="#Resolver.get_table-483"><span class="linenos">483</span></a> <span class="n">node</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">parent</span>
|
||||
</span><span id="Resolver.get_table-484"><a href="#Resolver.get_table-484"><span class="linenos">484</span></a>
|
||||
</span><span id="Resolver.get_table-485"><a href="#Resolver.get_table-485"><span class="linenos">485</span></a> <span class="n">node_alias</span> <span class="o">=</span> <span class="n">node</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"alias"</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-486"><a href="#Resolver.get_table-486"><span class="linenos">486</span></a> <span class="k">if</span> <span class="n">node_alias</span><span class="p">:</span>
|
||||
</span><span id="Resolver.get_table-487"><a href="#Resolver.get_table-487"><span class="linenos">487</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">node_alias</span><span class="o">.</span><span class="n">this</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-488"><a href="#Resolver.get_table-488"><span class="linenos">488</span></a>
|
||||
</span><span id="Resolver.get_table-489"><a href="#Resolver.get_table-489"><span class="linenos">489</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_table-479"><a href="#Resolver.get_table-479"><span class="linenos">479</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="n">table_name</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1084,22 +1064,22 @@
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#Resolver.get_source_columns"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Resolver.get_source_columns-500"><a href="#Resolver.get_source_columns-500"><span class="linenos">500</span></a> <span class="k">def</span> <span class="nf">get_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">only_visible</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
|
||||
</span><span id="Resolver.get_source_columns-501"><a href="#Resolver.get_source_columns-501"><span class="linenos">501</span></a><span class="w"> </span><span class="sd">"""Resolve the source columns for a given source `name`"""</span>
|
||||
</span><span id="Resolver.get_source_columns-502"><a href="#Resolver.get_source_columns-502"><span class="linenos">502</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="p">:</span>
|
||||
</span><span id="Resolver.get_source_columns-503"><a href="#Resolver.get_source_columns-503"><span class="linenos">503</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown table: </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_source_columns-504"><a href="#Resolver.get_source_columns-504"><span class="linenos">504</span></a>
|
||||
</span><span id="Resolver.get_source_columns-505"><a href="#Resolver.get_source_columns-505"><span class="linenos">505</span></a> <span class="n">source</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
|
||||
</span><span id="Resolver.get_source_columns-506"><a href="#Resolver.get_source_columns-506"><span class="linenos">506</span></a>
|
||||
</span><span id="Resolver.get_source_columns-507"><a href="#Resolver.get_source_columns-507"><span class="linenos">507</span></a> <span class="c1"># If referencing a table, return the columns from the schema</span>
|
||||
</span><span id="Resolver.get_source_columns-508"><a href="#Resolver.get_source_columns-508"><span class="linenos">508</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="Resolver.get_source_columns-509"><a href="#Resolver.get_source_columns-509"><span class="linenos">509</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span><span class="o">.</span><span class="n">column_names</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">only_visible</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_source_columns-510"><a href="#Resolver.get_source_columns-510"><span class="linenos">510</span></a>
|
||||
</span><span id="Resolver.get_source_columns-511"><a href="#Resolver.get_source_columns-511"><span class="linenos">511</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">):</span>
|
||||
</span><span id="Resolver.get_source_columns-512"><a href="#Resolver.get_source_columns-512"><span class="linenos">512</span></a> <span class="k">return</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">alias_column_names</span>
|
||||
</span><span id="Resolver.get_source_columns-513"><a href="#Resolver.get_source_columns-513"><span class="linenos">513</span></a>
|
||||
</span><span id="Resolver.get_source_columns-514"><a href="#Resolver.get_source_columns-514"><span class="linenos">514</span></a> <span class="c1"># Otherwise, if referencing another scope, return that scope's named selects</span>
|
||||
</span><span id="Resolver.get_source_columns-515"><a href="#Resolver.get_source_columns-515"><span class="linenos">515</span></a> <span class="k">return</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="Resolver.get_source_columns-490"><a href="#Resolver.get_source_columns-490"><span class="linenos">490</span></a> <span class="k">def</span> <span class="nf">get_source_columns</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">only_visible</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
|
||||
</span><span id="Resolver.get_source_columns-491"><a href="#Resolver.get_source_columns-491"><span class="linenos">491</span></a><span class="w"> </span><span class="sd">"""Resolve the source columns for a given source `name`"""</span>
|
||||
</span><span id="Resolver.get_source_columns-492"><a href="#Resolver.get_source_columns-492"><span class="linenos">492</span></a> <span class="k">if</span> <span class="n">name</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="p">:</span>
|
||||
</span><span id="Resolver.get_source_columns-493"><a href="#Resolver.get_source_columns-493"><span class="linenos">493</span></a> <span class="k">raise</span> <span class="n">OptimizeError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown table: </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_source_columns-494"><a href="#Resolver.get_source_columns-494"><span class="linenos">494</span></a>
|
||||
</span><span id="Resolver.get_source_columns-495"><a href="#Resolver.get_source_columns-495"><span class="linenos">495</span></a> <span class="n">source</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">scope</span><span class="o">.</span><span class="n">sources</span><span class="p">[</span><span class="n">name</span><span class="p">]</span>
|
||||
</span><span id="Resolver.get_source_columns-496"><a href="#Resolver.get_source_columns-496"><span class="linenos">496</span></a>
|
||||
</span><span id="Resolver.get_source_columns-497"><a href="#Resolver.get_source_columns-497"><span class="linenos">497</span></a> <span class="c1"># If referencing a table, return the columns from the schema</span>
|
||||
</span><span id="Resolver.get_source_columns-498"><a href="#Resolver.get_source_columns-498"><span class="linenos">498</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">):</span>
|
||||
</span><span id="Resolver.get_source_columns-499"><a href="#Resolver.get_source_columns-499"><span class="linenos">499</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">schema</span><span class="o">.</span><span class="n">column_names</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">only_visible</span><span class="p">)</span>
|
||||
</span><span id="Resolver.get_source_columns-500"><a href="#Resolver.get_source_columns-500"><span class="linenos">500</span></a>
|
||||
</span><span id="Resolver.get_source_columns-501"><a href="#Resolver.get_source_columns-501"><span class="linenos">501</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="p">,</span> <span class="n">Scope</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Values</span><span class="p">):</span>
|
||||
</span><span id="Resolver.get_source_columns-502"><a href="#Resolver.get_source_columns-502"><span class="linenos">502</span></a> <span class="k">return</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">alias_column_names</span>
|
||||
</span><span id="Resolver.get_source_columns-503"><a href="#Resolver.get_source_columns-503"><span class="linenos">503</span></a>
|
||||
</span><span id="Resolver.get_source_columns-504"><a href="#Resolver.get_source_columns-504"><span class="linenos">504</span></a> <span class="c1"># Otherwise, if referencing another scope, return that scope's named selects</span>
|
||||
</span><span id="Resolver.get_source_columns-505"><a href="#Resolver.get_source_columns-505"><span class="linenos">505</span></a> <span class="k">return</span> <span class="n">source</span><span class="o">.</span><span class="n">expression</span><span class="o">.</span><span class="n">named_selects</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
16098
docs/sqlglot/parser.html
16098
docs/sqlglot/parser.html
File diff suppressed because it is too large
Load diff
|
|
@ -423,7 +423,7 @@
|
|||
</span><span id="L-302"><a href="#L-302"><span class="linenos">302</span></a>
|
||||
</span><span id="L-303"><a href="#L-303"><span class="linenos">303</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="p">:</span>
|
||||
</span><span id="L-304"><a href="#L-304"><span class="linenos">304</span></a> <span class="n">source_key</span><span class="p">,</span> <span class="n">join_key</span><span class="p">,</span> <span class="n">condition</span> <span class="o">=</span> <span class="n">join_condition</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
</span><span id="L-305"><a href="#L-305"><span class="linenos">305</span></a> <span class="n">step</span><span class="o">.</span><span class="n">joins</span><span class="p">[</span><span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="L-305"><a href="#L-305"><span class="linenos">305</span></a> <span class="n">step</span><span class="o">.</span><span class="n">joins</span><span class="p">[</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="L-306"><a href="#L-306"><span class="linenos">306</span></a> <span class="s2">"side"</span><span class="p">:</span> <span class="n">join</span><span class="o">.</span><span class="n">side</span><span class="p">,</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="L-307"><a href="#L-307"><span class="linenos">307</span></a> <span class="s2">"join_key"</span><span class="p">:</span> <span class="n">join_key</span><span class="p">,</span>
|
||||
</span><span id="L-308"><a href="#L-308"><span class="linenos">308</span></a> <span class="s2">"source_key"</span><span class="p">:</span> <span class="n">source_key</span><span class="p">,</span>
|
||||
|
|
@ -1316,7 +1316,7 @@ Projections:</li>
|
|||
</span><span id="Join-303"><a href="#Join-303"><span class="linenos">303</span></a>
|
||||
</span><span id="Join-304"><a href="#Join-304"><span class="linenos">304</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="p">:</span>
|
||||
</span><span id="Join-305"><a href="#Join-305"><span class="linenos">305</span></a> <span class="n">source_key</span><span class="p">,</span> <span class="n">join_key</span><span class="p">,</span> <span class="n">condition</span> <span class="o">=</span> <span class="n">join_condition</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
</span><span id="Join-306"><a href="#Join-306"><span class="linenos">306</span></a> <span class="n">step</span><span class="o">.</span><span class="n">joins</span><span class="p">[</span><span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="Join-306"><a href="#Join-306"><span class="linenos">306</span></a> <span class="n">step</span><span class="o">.</span><span class="n">joins</span><span class="p">[</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="Join-307"><a href="#Join-307"><span class="linenos">307</span></a> <span class="s2">"side"</span><span class="p">:</span> <span class="n">join</span><span class="o">.</span><span class="n">side</span><span class="p">,</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="Join-308"><a href="#Join-308"><span class="linenos">308</span></a> <span class="s2">"join_key"</span><span class="p">:</span> <span class="n">join_key</span><span class="p">,</span>
|
||||
</span><span id="Join-309"><a href="#Join-309"><span class="linenos">309</span></a> <span class="s2">"source_key"</span><span class="p">:</span> <span class="n">source_key</span><span class="p">,</span>
|
||||
|
|
@ -1363,7 +1363,7 @@ Projections:</li>
|
|||
</span><span id="Join.from_joins-303"><a href="#Join.from_joins-303"><span class="linenos">303</span></a>
|
||||
</span><span id="Join.from_joins-304"><a href="#Join.from_joins-304"><span class="linenos">304</span></a> <span class="k">for</span> <span class="n">join</span> <span class="ow">in</span> <span class="n">joins</span><span class="p">:</span>
|
||||
</span><span id="Join.from_joins-305"><a href="#Join.from_joins-305"><span class="linenos">305</span></a> <span class="n">source_key</span><span class="p">,</span> <span class="n">join_key</span><span class="p">,</span> <span class="n">condition</span> <span class="o">=</span> <span class="n">join_condition</span><span class="p">(</span><span class="n">join</span><span class="p">)</span>
|
||||
</span><span id="Join.from_joins-306"><a href="#Join.from_joins-306"><span class="linenos">306</span></a> <span class="n">step</span><span class="o">.</span><span class="n">joins</span><span class="p">[</span><span class="n">join</span><span class="o">.</span><span class="n">this</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="Join.from_joins-306"><a href="#Join.from_joins-306"><span class="linenos">306</span></a> <span class="n">step</span><span class="o">.</span><span class="n">joins</span><span class="p">[</span><span class="n">join</span><span class="o">.</span><span class="n">alias_or_name</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span>
|
||||
</span><span id="Join.from_joins-307"><a href="#Join.from_joins-307"><span class="linenos">307</span></a> <span class="s2">"side"</span><span class="p">:</span> <span class="n">join</span><span class="o">.</span><span class="n">side</span><span class="p">,</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="Join.from_joins-308"><a href="#Join.from_joins-308"><span class="linenos">308</span></a> <span class="s2">"join_key"</span><span class="p">:</span> <span class="n">join_key</span><span class="p">,</span>
|
||||
</span><span id="Join.from_joins-309"><a href="#Join.from_joins-309"><span class="linenos">309</span></a> <span class="s2">"source_key"</span><span class="p">:</span> <span class="n">source_key</span><span class="p">,</span>
|
||||
|
|
|
|||
|
|
@ -139,7 +139,7 @@
|
|||
</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="kn">import</span> <span class="nn">sqlglot</span>
|
||||
</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">from</span> <span class="nn">sqlglot</span> <span class="kn">import</span> <span class="n">expressions</span> <span class="k">as</span> <span class="n">exp</span>
|
||||
</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">from</span> <span class="nn">sqlglot._typing</span> <span class="kn">import</span> <span class="n">T</span>
|
||||
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">RESOLVES_IDENTIFIERS_AS_UPPERCASE</span>
|
||||
</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">from</span> <span class="nn">sqlglot.dialects.dialect</span> <span class="kn">import</span> <span class="n">Dialect</span>
|
||||
</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="kn">from</span> <span class="nn">sqlglot.errors</span> <span class="kn">import</span> <span class="n">ParseError</span><span class="p">,</span> <span class="n">SchemaError</span>
|
||||
</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="kn">from</span> <span class="nn">sqlglot.helper</span> <span class="kn">import</span> <span class="n">dict_depth</span>
|
||||
</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a><span class="kn">from</span> <span class="nn">sqlglot.trie</span> <span class="kn">import</span> <span class="n">in_trie</span><span class="p">,</span> <span class="n">new_trie</span>
|
||||
|
|
@ -422,7 +422,7 @@
|
|||
</span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a>
|
||||
</span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a> <span class="k">def</span> <span class="nf">_normalize</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a><span class="sd"> Converts all identifiers in the schema into lowercase, unless they're quoted.</span>
|
||||
</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a><span class="sd"> Normalizes all identifiers in the schema.</span>
|
||||
</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a>
|
||||
</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a><span class="sd"> schema: the schema to normalize.</span>
|
||||
|
|
@ -437,177 +437,183 @@
|
|||
</span><span id="L-304"><a href="#L-304"><span class="linenos">304</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="n">nested_get</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="o">*</span><span class="nb">zip</span><span class="p">(</span><span class="n">keys</span><span class="p">,</span> <span class="n">keys</span><span class="p">))</span>
|
||||
</span><span id="L-305"><a href="#L-305"><span class="linenos">305</span></a> <span class="k">assert</span> <span class="n">columns</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
|
||||
</span><span id="L-306"><a href="#L-306"><span class="linenos">306</span></a>
|
||||
</span><span id="L-307"><a href="#L-307"><span class="linenos">307</span></a> <span class="n">normalized_keys</span> <span class="o">=</span> <span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">_normalize_name</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">keys</span><span class="p">]</span>
|
||||
</span><span id="L-308"><a href="#L-308"><span class="linenos">308</span></a> <span class="k">for</span> <span class="n">column_name</span><span class="p">,</span> <span class="n">column_type</span> <span class="ow">in</span> <span class="n">columns</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-309"><a href="#L-309"><span class="linenos">309</span></a> <span class="n">nested_set</span><span class="p">(</span>
|
||||
</span><span id="L-310"><a href="#L-310"><span class="linenos">310</span></a> <span class="n">normalized_mapping</span><span class="p">,</span>
|
||||
</span><span id="L-311"><a href="#L-311"><span class="linenos">311</span></a> <span class="n">normalized_keys</span> <span class="o">+</span> <span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">_normalize_name</span><span class="p">(</span><span class="n">column_name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)],</span>
|
||||
</span><span id="L-312"><a href="#L-312"><span class="linenos">312</span></a> <span class="n">column_type</span><span class="p">,</span>
|
||||
</span><span id="L-313"><a href="#L-313"><span class="linenos">313</span></a> <span class="p">)</span>
|
||||
</span><span id="L-314"><a href="#L-314"><span class="linenos">314</span></a>
|
||||
</span><span id="L-315"><a href="#L-315"><span class="linenos">315</span></a> <span class="k">return</span> <span class="n">normalized_mapping</span>
|
||||
</span><span id="L-307"><a href="#L-307"><span class="linenos">307</span></a> <span class="n">normalized_keys</span> <span class="o">=</span> <span class="p">[</span>
|
||||
</span><span id="L-308"><a href="#L-308"><span class="linenos">308</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_normalize_name</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">,</span> <span class="n">is_table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">keys</span>
|
||||
</span><span id="L-309"><a href="#L-309"><span class="linenos">309</span></a> <span class="p">]</span>
|
||||
</span><span id="L-310"><a href="#L-310"><span class="linenos">310</span></a> <span class="k">for</span> <span class="n">column_name</span><span class="p">,</span> <span class="n">column_type</span> <span class="ow">in</span> <span class="n">columns</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-311"><a href="#L-311"><span class="linenos">311</span></a> <span class="n">nested_set</span><span class="p">(</span>
|
||||
</span><span id="L-312"><a href="#L-312"><span class="linenos">312</span></a> <span class="n">normalized_mapping</span><span class="p">,</span>
|
||||
</span><span id="L-313"><a href="#L-313"><span class="linenos">313</span></a> <span class="n">normalized_keys</span> <span class="o">+</span> <span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">_normalize_name</span><span class="p">(</span><span class="n">column_name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)],</span>
|
||||
</span><span id="L-314"><a href="#L-314"><span class="linenos">314</span></a> <span class="n">column_type</span><span class="p">,</span>
|
||||
</span><span id="L-315"><a href="#L-315"><span class="linenos">315</span></a> <span class="p">)</span>
|
||||
</span><span id="L-316"><a href="#L-316"><span class="linenos">316</span></a>
|
||||
</span><span id="L-317"><a href="#L-317"><span class="linenos">317</span></a> <span class="k">def</span> <span class="nf">_normalize_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">:</span>
|
||||
</span><span id="L-318"><a href="#L-318"><span class="linenos">318</span></a> <span class="n">normalized_table</span> <span class="o">=</span> <span class="n">table</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
</span><span id="L-319"><a href="#L-319"><span class="linenos">319</span></a>
|
||||
</span><span id="L-320"><a href="#L-320"><span class="linenos">320</span></a> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="n">TABLE_ARGS</span><span class="p">:</span>
|
||||
</span><span id="L-321"><a href="#L-321"><span class="linenos">321</span></a> <span class="n">value</span> <span class="o">=</span> <span class="n">normalized_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span>
|
||||
</span><span id="L-322"><a href="#L-322"><span class="linenos">322</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="p">(</span><span class="nb">str</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)):</span>
|
||||
</span><span id="L-323"><a href="#L-323"><span class="linenos">323</span></a> <span class="n">normalized_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-324"><a href="#L-324"><span class="linenos">324</span></a> <span class="n">arg</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_normalize_name</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">))</span>
|
||||
</span><span id="L-325"><a href="#L-325"><span class="linenos">325</span></a> <span class="p">)</span>
|
||||
</span><span id="L-326"><a href="#L-326"><span class="linenos">326</span></a>
|
||||
</span><span id="L-327"><a href="#L-327"><span class="linenos">327</span></a> <span class="k">return</span> <span class="n">normalized_table</span>
|
||||
</span><span id="L-328"><a href="#L-328"><span class="linenos">328</span></a>
|
||||
</span><span id="L-329"><a href="#L-329"><span class="linenos">329</span></a> <span class="k">def</span> <span class="nf">_normalize_name</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="L-330"><a href="#L-330"><span class="linenos">330</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span>
|
||||
</span><span id="L-317"><a href="#L-317"><span class="linenos">317</span></a> <span class="k">return</span> <span class="n">normalized_mapping</span>
|
||||
</span><span id="L-318"><a href="#L-318"><span class="linenos">318</span></a>
|
||||
</span><span id="L-319"><a href="#L-319"><span class="linenos">319</span></a> <span class="k">def</span> <span class="nf">_normalize_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">:</span>
|
||||
</span><span id="L-320"><a href="#L-320"><span class="linenos">320</span></a> <span class="n">normalized_table</span> <span class="o">=</span> <span class="n">table</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
</span><span id="L-321"><a href="#L-321"><span class="linenos">321</span></a>
|
||||
</span><span id="L-322"><a href="#L-322"><span class="linenos">322</span></a> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="n">TABLE_ARGS</span><span class="p">:</span>
|
||||
</span><span id="L-323"><a href="#L-323"><span class="linenos">323</span></a> <span class="n">value</span> <span class="o">=</span> <span class="n">normalized_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span>
|
||||
</span><span id="L-324"><a href="#L-324"><span class="linenos">324</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="p">(</span><span class="nb">str</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)):</span>
|
||||
</span><span id="L-325"><a href="#L-325"><span class="linenos">325</span></a> <span class="n">normalized_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="L-326"><a href="#L-326"><span class="linenos">326</span></a> <span class="n">arg</span><span class="p">,</span>
|
||||
</span><span id="L-327"><a href="#L-327"><span class="linenos">327</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_normalize_name</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">is_table</span><span class="o">=</span><span class="kc">True</span><span class="p">)),</span>
|
||||
</span><span id="L-328"><a href="#L-328"><span class="linenos">328</span></a> <span class="p">)</span>
|
||||
</span><span id="L-329"><a href="#L-329"><span class="linenos">329</span></a>
|
||||
</span><span id="L-330"><a href="#L-330"><span class="linenos">330</span></a> <span class="k">return</span> <span class="n">normalized_table</span>
|
||||
</span><span id="L-331"><a href="#L-331"><span class="linenos">331</span></a>
|
||||
</span><span id="L-332"><a href="#L-332"><span class="linenos">332</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="L-333"><a href="#L-333"><span class="linenos">333</span></a> <span class="n">identifier</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">maybe_parse</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span>
|
||||
</span><span id="L-334"><a href="#L-334"><span class="linenos">334</span></a> <span class="k">except</span> <span class="n">ParseError</span><span class="p">:</span>
|
||||
</span><span id="L-335"><a href="#L-335"><span class="linenos">335</span></a> <span class="k">return</span> <span class="n">name</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">name</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-332"><a href="#L-332"><span class="linenos">332</span></a> <span class="k">def</span> <span class="nf">_normalize_name</span><span class="p">(</span>
|
||||
</span><span id="L-333"><a href="#L-333"><span class="linenos">333</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="n">is_table</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="L-334"><a href="#L-334"><span class="linenos">334</span></a> <span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="L-335"><a href="#L-335"><span class="linenos">335</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span>
|
||||
</span><span id="L-336"><a href="#L-336"><span class="linenos">336</span></a>
|
||||
</span><span id="L-337"><a href="#L-337"><span class="linenos">337</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">identifier</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-338"><a href="#L-338"><span class="linenos">338</span></a>
|
||||
</span><span id="L-339"><a href="#L-339"><span class="linenos">339</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">normalize</span> <span class="ow">or</span> <span class="n">identifier</span><span class="o">.</span><span class="n">quoted</span><span class="p">:</span>
|
||||
</span><span id="L-340"><a href="#L-340"><span class="linenos">340</span></a> <span class="k">return</span> <span class="n">name</span>
|
||||
</span><span id="L-337"><a href="#L-337"><span class="linenos">337</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="L-338"><a href="#L-338"><span class="linenos">338</span></a> <span class="n">identifier</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">maybe_parse</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span>
|
||||
</span><span id="L-339"><a href="#L-339"><span class="linenos">339</span></a> <span class="k">except</span> <span class="n">ParseError</span><span class="p">:</span>
|
||||
</span><span id="L-340"><a href="#L-340"><span class="linenos">340</span></a> <span class="k">return</span> <span class="n">name</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">name</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-341"><a href="#L-341"><span class="linenos">341</span></a>
|
||||
</span><span id="L-342"><a href="#L-342"><span class="linenos">342</span></a> <span class="k">return</span> <span class="n">name</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span> <span class="k">if</span> <span class="n">dialect</span> <span class="ow">in</span> <span class="n">RESOLVES_IDENTIFIERS_AS_UPPERCASE</span> <span class="k">else</span> <span class="n">name</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span>
|
||||
</span><span id="L-343"><a href="#L-343"><span class="linenos">343</span></a>
|
||||
</span><span id="L-344"><a href="#L-344"><span class="linenos">344</span></a> <span class="k">def</span> <span class="nf">_depth</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">int</span><span class="p">:</span>
|
||||
</span><span id="L-345"><a href="#L-345"><span class="linenos">345</span></a> <span class="c1"># The columns themselves are a mapping, but we don't want to include those</span>
|
||||
</span><span id="L-346"><a href="#L-346"><span class="linenos">346</span></a> <span class="k">return</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">_depth</span><span class="p">()</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
</span><span id="L-347"><a href="#L-347"><span class="linenos">347</span></a>
|
||||
</span><span id="L-348"><a href="#L-348"><span class="linenos">348</span></a> <span class="k">def</span> <span class="nf">_ensure_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span> <span class="o">|</span> <span class="nb">str</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">:</span>
|
||||
</span><span id="L-349"><a href="#L-349"><span class="linenos">349</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">maybe_parse</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-350"><a href="#L-350"><span class="linenos">350</span></a>
|
||||
</span><span id="L-351"><a href="#L-351"><span class="linenos">351</span></a> <span class="k">def</span> <span class="nf">_to_data_type</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">schema_type</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="p">:</span>
|
||||
</span><span id="L-352"><a href="#L-352"><span class="linenos">352</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-353"><a href="#L-353"><span class="linenos">353</span></a><span class="sd"> Convert a type represented as a string to the corresponding `sqlglot.exp.DataType` object.</span>
|
||||
</span><span id="L-354"><a href="#L-354"><span class="linenos">354</span></a>
|
||||
</span><span id="L-355"><a href="#L-355"><span class="linenos">355</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-356"><a href="#L-356"><span class="linenos">356</span></a><span class="sd"> schema_type: the type we want to convert.</span>
|
||||
</span><span id="L-357"><a href="#L-357"><span class="linenos">357</span></a><span class="sd"> dialect: the SQL dialect that will be used to parse `schema_type`, if needed.</span>
|
||||
</span><span id="L-358"><a href="#L-358"><span class="linenos">358</span></a>
|
||||
</span><span id="L-359"><a href="#L-359"><span class="linenos">359</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-360"><a href="#L-360"><span class="linenos">360</span></a><span class="sd"> The resulting expression type.</span>
|
||||
</span><span id="L-361"><a href="#L-361"><span class="linenos">361</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-362"><a href="#L-362"><span class="linenos">362</span></a> <span class="k">if</span> <span class="n">schema_type</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">:</span>
|
||||
</span><span id="L-363"><a href="#L-363"><span class="linenos">363</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span>
|
||||
</span><span id="L-342"><a href="#L-342"><span class="linenos">342</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">identifier</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-343"><a href="#L-343"><span class="linenos">343</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">normalize</span><span class="p">:</span>
|
||||
</span><span id="L-344"><a href="#L-344"><span class="linenos">344</span></a> <span class="k">return</span> <span class="n">name</span>
|
||||
</span><span id="L-345"><a href="#L-345"><span class="linenos">345</span></a>
|
||||
</span><span id="L-346"><a href="#L-346"><span class="linenos">346</span></a> <span class="c1"># This can be useful for normalize_identifier</span>
|
||||
</span><span id="L-347"><a href="#L-347"><span class="linenos">347</span></a> <span class="n">identifier</span><span class="o">.</span><span class="n">meta</span><span class="p">[</span><span class="s2">"is_table"</span><span class="p">]</span> <span class="o">=</span> <span class="n">is_table</span>
|
||||
</span><span id="L-348"><a href="#L-348"><span class="linenos">348</span></a> <span class="k">return</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">identifier</span><span class="p">)</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="L-349"><a href="#L-349"><span class="linenos">349</span></a>
|
||||
</span><span id="L-350"><a href="#L-350"><span class="linenos">350</span></a> <span class="k">def</span> <span class="nf">_depth</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">int</span><span class="p">:</span>
|
||||
</span><span id="L-351"><a href="#L-351"><span class="linenos">351</span></a> <span class="c1"># The columns themselves are a mapping, but we don't want to include those</span>
|
||||
</span><span id="L-352"><a href="#L-352"><span class="linenos">352</span></a> <span class="k">return</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">_depth</span><span class="p">()</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
</span><span id="L-353"><a href="#L-353"><span class="linenos">353</span></a>
|
||||
</span><span id="L-354"><a href="#L-354"><span class="linenos">354</span></a> <span class="k">def</span> <span class="nf">_ensure_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span> <span class="o">|</span> <span class="nb">str</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">:</span>
|
||||
</span><span id="L-355"><a href="#L-355"><span class="linenos">355</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">maybe_parse</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-356"><a href="#L-356"><span class="linenos">356</span></a>
|
||||
</span><span id="L-357"><a href="#L-357"><span class="linenos">357</span></a> <span class="k">def</span> <span class="nf">_to_data_type</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">schema_type</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="p">:</span>
|
||||
</span><span id="L-358"><a href="#L-358"><span class="linenos">358</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-359"><a href="#L-359"><span class="linenos">359</span></a><span class="sd"> Convert a type represented as a string to the corresponding `sqlglot.exp.DataType` object.</span>
|
||||
</span><span id="L-360"><a href="#L-360"><span class="linenos">360</span></a>
|
||||
</span><span id="L-361"><a href="#L-361"><span class="linenos">361</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-362"><a href="#L-362"><span class="linenos">362</span></a><span class="sd"> schema_type: the type we want to convert.</span>
|
||||
</span><span id="L-363"><a href="#L-363"><span class="linenos">363</span></a><span class="sd"> dialect: the SQL dialect that will be used to parse `schema_type`, if needed.</span>
|
||||
</span><span id="L-364"><a href="#L-364"><span class="linenos">364</span></a>
|
||||
</span><span id="L-365"><a href="#L-365"><span class="linenos">365</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="L-366"><a href="#L-366"><span class="linenos">366</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">build</span><span class="p">(</span><span class="n">schema_type</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-367"><a href="#L-367"><span class="linenos">367</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span> <span class="o">=</span> <span class="n">expression</span>
|
||||
</span><span id="L-368"><a href="#L-368"><span class="linenos">368</span></a> <span class="k">except</span> <span class="ne">AttributeError</span><span class="p">:</span>
|
||||
</span><span id="L-369"><a href="#L-369"><span class="linenos">369</span></a> <span class="n">in_dialect</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">" in dialect </span><span class="si">{</span><span class="n">dialect</span><span class="si">}</span><span class="s2">"</span> <span class="k">if</span> <span class="n">dialect</span> <span class="k">else</span> <span class="s2">""</span>
|
||||
</span><span id="L-370"><a href="#L-370"><span class="linenos">370</span></a> <span class="k">raise</span> <span class="n">SchemaError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Failed to build type '</span><span class="si">{</span><span class="n">schema_type</span><span class="si">}</span><span class="s2">'</span><span class="si">{</span><span class="n">in_dialect</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
|
||||
</span><span id="L-371"><a href="#L-371"><span class="linenos">371</span></a>
|
||||
</span><span id="L-372"><a href="#L-372"><span class="linenos">372</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span>
|
||||
</span><span id="L-373"><a href="#L-373"><span class="linenos">373</span></a>
|
||||
</span><span id="L-374"><a href="#L-374"><span class="linenos">374</span></a>
|
||||
</span><span id="L-375"><a href="#L-375"><span class="linenos">375</span></a><span class="k">def</span> <span class="nf">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">:</span> <span class="n">Schema</span> <span class="o">|</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">Schema</span><span class="p">:</span>
|
||||
</span><span id="L-376"><a href="#L-376"><span class="linenos">376</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">Schema</span><span class="p">):</span>
|
||||
</span><span id="L-377"><a href="#L-377"><span class="linenos">377</span></a> <span class="k">return</span> <span class="n">schema</span>
|
||||
</span><span id="L-378"><a href="#L-378"><span class="linenos">378</span></a>
|
||||
</span><span id="L-379"><a href="#L-379"><span class="linenos">379</span></a> <span class="k">return</span> <span class="n">MappingSchema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
</span><span id="L-365"><a href="#L-365"><span class="linenos">365</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-366"><a href="#L-366"><span class="linenos">366</span></a><span class="sd"> The resulting expression type.</span>
|
||||
</span><span id="L-367"><a href="#L-367"><span class="linenos">367</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-368"><a href="#L-368"><span class="linenos">368</span></a> <span class="k">if</span> <span class="n">schema_type</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">:</span>
|
||||
</span><span id="L-369"><a href="#L-369"><span class="linenos">369</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span>
|
||||
</span><span id="L-370"><a href="#L-370"><span class="linenos">370</span></a>
|
||||
</span><span id="L-371"><a href="#L-371"><span class="linenos">371</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="L-372"><a href="#L-372"><span class="linenos">372</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">build</span><span class="p">(</span><span class="n">schema_type</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="L-373"><a href="#L-373"><span class="linenos">373</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span> <span class="o">=</span> <span class="n">expression</span>
|
||||
</span><span id="L-374"><a href="#L-374"><span class="linenos">374</span></a> <span class="k">except</span> <span class="ne">AttributeError</span><span class="p">:</span>
|
||||
</span><span id="L-375"><a href="#L-375"><span class="linenos">375</span></a> <span class="n">in_dialect</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">" in dialect </span><span class="si">{</span><span class="n">dialect</span><span class="si">}</span><span class="s2">"</span> <span class="k">if</span> <span class="n">dialect</span> <span class="k">else</span> <span class="s2">""</span>
|
||||
</span><span id="L-376"><a href="#L-376"><span class="linenos">376</span></a> <span class="k">raise</span> <span class="n">SchemaError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Failed to build type '</span><span class="si">{</span><span class="n">schema_type</span><span class="si">}</span><span class="s2">'</span><span class="si">{</span><span class="n">in_dialect</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
|
||||
</span><span id="L-377"><a href="#L-377"><span class="linenos">377</span></a>
|
||||
</span><span id="L-378"><a href="#L-378"><span class="linenos">378</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span>
|
||||
</span><span id="L-379"><a href="#L-379"><span class="linenos">379</span></a>
|
||||
</span><span id="L-380"><a href="#L-380"><span class="linenos">380</span></a>
|
||||
</span><span id="L-381"><a href="#L-381"><span class="linenos">381</span></a>
|
||||
</span><span id="L-382"><a href="#L-382"><span class="linenos">382</span></a><span class="k">def</span> <span class="nf">ensure_column_mapping</span><span class="p">(</span><span class="n">mapping</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">ColumnMapping</span><span class="p">])</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="L-383"><a href="#L-383"><span class="linenos">383</span></a> <span class="k">if</span> <span class="n">mapping</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-384"><a href="#L-384"><span class="linenos">384</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-385"><a href="#L-385"><span class="linenos">385</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">dict</span><span class="p">):</span>
|
||||
</span><span id="L-386"><a href="#L-386"><span class="linenos">386</span></a> <span class="k">return</span> <span class="n">mapping</span>
|
||||
</span><span id="L-387"><a href="#L-387"><span class="linenos">387</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
|
||||
</span><span id="L-388"><a href="#L-388"><span class="linenos">388</span></a> <span class="n">col_name_type_strs</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">","</span><span class="p">)]</span>
|
||||
</span><span id="L-389"><a href="#L-389"><span class="linenos">389</span></a> <span class="k">return</span> <span class="p">{</span>
|
||||
</span><span id="L-390"><a href="#L-390"><span class="linenos">390</span></a> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
|
||||
</span><span id="L-391"><a href="#L-391"><span class="linenos">391</span></a> <span class="k">for</span> <span class="n">name_type_str</span> <span class="ow">in</span> <span class="n">col_name_type_strs</span>
|
||||
</span><span id="L-392"><a href="#L-392"><span class="linenos">392</span></a> <span class="p">}</span>
|
||||
</span><span id="L-393"><a href="#L-393"><span class="linenos">393</span></a> <span class="c1"># Check if mapping looks like a DataFrame StructType</span>
|
||||
</span><span id="L-394"><a href="#L-394"><span class="linenos">394</span></a> <span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="s2">"simpleString"</span><span class="p">):</span>
|
||||
</span><span id="L-395"><a href="#L-395"><span class="linenos">395</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">struct_field</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="n">struct_field</span><span class="o">.</span><span class="n">dataType</span><span class="o">.</span><span class="n">simpleString</span><span class="p">()</span> <span class="k">for</span> <span class="n">struct_field</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="L-396"><a href="#L-396"><span class="linenos">396</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
|
||||
</span><span id="L-397"><a href="#L-397"><span class="linenos">397</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="L-398"><a href="#L-398"><span class="linenos">398</span></a>
|
||||
</span><span id="L-399"><a href="#L-399"><span class="linenos">399</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Invalid mapping provided: </span><span class="si">{</span><span class="nb">type</span><span class="p">(</span><span class="n">mapping</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="L-400"><a href="#L-400"><span class="linenos">400</span></a>
|
||||
</span><span id="L-401"><a href="#L-401"><span class="linenos">401</span></a>
|
||||
</span><span id="L-402"><a href="#L-402"><span class="linenos">402</span></a><span class="k">def</span> <span class="nf">flatten_schema</span><span class="p">(</span>
|
||||
</span><span id="L-403"><a href="#L-403"><span class="linenos">403</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">depth</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-404"><a href="#L-404"><span class="linenos">404</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
|
||||
</span><span id="L-405"><a href="#L-405"><span class="linenos">405</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-406"><a href="#L-406"><span class="linenos">406</span></a> <span class="n">keys</span> <span class="o">=</span> <span class="n">keys</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="L-381"><a href="#L-381"><span class="linenos">381</span></a><span class="k">def</span> <span class="nf">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">:</span> <span class="n">Schema</span> <span class="o">|</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">Schema</span><span class="p">:</span>
|
||||
</span><span id="L-382"><a href="#L-382"><span class="linenos">382</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">Schema</span><span class="p">):</span>
|
||||
</span><span id="L-383"><a href="#L-383"><span class="linenos">383</span></a> <span class="k">return</span> <span class="n">schema</span>
|
||||
</span><span id="L-384"><a href="#L-384"><span class="linenos">384</span></a>
|
||||
</span><span id="L-385"><a href="#L-385"><span class="linenos">385</span></a> <span class="k">return</span> <span class="n">MappingSchema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
</span><span id="L-386"><a href="#L-386"><span class="linenos">386</span></a>
|
||||
</span><span id="L-387"><a href="#L-387"><span class="linenos">387</span></a>
|
||||
</span><span id="L-388"><a href="#L-388"><span class="linenos">388</span></a><span class="k">def</span> <span class="nf">ensure_column_mapping</span><span class="p">(</span><span class="n">mapping</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">ColumnMapping</span><span class="p">])</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="L-389"><a href="#L-389"><span class="linenos">389</span></a> <span class="k">if</span> <span class="n">mapping</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-390"><a href="#L-390"><span class="linenos">390</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="L-391"><a href="#L-391"><span class="linenos">391</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">dict</span><span class="p">):</span>
|
||||
</span><span id="L-392"><a href="#L-392"><span class="linenos">392</span></a> <span class="k">return</span> <span class="n">mapping</span>
|
||||
</span><span id="L-393"><a href="#L-393"><span class="linenos">393</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
|
||||
</span><span id="L-394"><a href="#L-394"><span class="linenos">394</span></a> <span class="n">col_name_type_strs</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">","</span><span class="p">)]</span>
|
||||
</span><span id="L-395"><a href="#L-395"><span class="linenos">395</span></a> <span class="k">return</span> <span class="p">{</span>
|
||||
</span><span id="L-396"><a href="#L-396"><span class="linenos">396</span></a> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
|
||||
</span><span id="L-397"><a href="#L-397"><span class="linenos">397</span></a> <span class="k">for</span> <span class="n">name_type_str</span> <span class="ow">in</span> <span class="n">col_name_type_strs</span>
|
||||
</span><span id="L-398"><a href="#L-398"><span class="linenos">398</span></a> <span class="p">}</span>
|
||||
</span><span id="L-399"><a href="#L-399"><span class="linenos">399</span></a> <span class="c1"># Check if mapping looks like a DataFrame StructType</span>
|
||||
</span><span id="L-400"><a href="#L-400"><span class="linenos">400</span></a> <span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="s2">"simpleString"</span><span class="p">):</span>
|
||||
</span><span id="L-401"><a href="#L-401"><span class="linenos">401</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">struct_field</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="n">struct_field</span><span class="o">.</span><span class="n">dataType</span><span class="o">.</span><span class="n">simpleString</span><span class="p">()</span> <span class="k">for</span> <span class="n">struct_field</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="L-402"><a href="#L-402"><span class="linenos">402</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
|
||||
</span><span id="L-403"><a href="#L-403"><span class="linenos">403</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="L-404"><a href="#L-404"><span class="linenos">404</span></a>
|
||||
</span><span id="L-405"><a href="#L-405"><span class="linenos">405</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Invalid mapping provided: </span><span class="si">{</span><span class="nb">type</span><span class="p">(</span><span class="n">mapping</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="L-406"><a href="#L-406"><span class="linenos">406</span></a>
|
||||
</span><span id="L-407"><a href="#L-407"><span class="linenos">407</span></a>
|
||||
</span><span id="L-408"><a href="#L-408"><span class="linenos">408</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">schema</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-409"><a href="#L-409"><span class="linenos">409</span></a> <span class="k">if</span> <span class="n">depth</span> <span class="o">>=</span> <span class="mi">2</span><span class="p">:</span>
|
||||
</span><span id="L-410"><a href="#L-410"><span class="linenos">410</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">flatten_schema</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">depth</span> <span class="o">-</span> <span class="mi">1</span><span class="p">,</span> <span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">]))</span>
|
||||
</span><span id="L-411"><a href="#L-411"><span class="linenos">411</span></a> <span class="k">elif</span> <span class="n">depth</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-412"><a href="#L-412"><span class="linenos">412</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">])</span>
|
||||
</span><span id="L-408"><a href="#L-408"><span class="linenos">408</span></a><span class="k">def</span> <span class="nf">flatten_schema</span><span class="p">(</span>
|
||||
</span><span id="L-409"><a href="#L-409"><span class="linenos">409</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">depth</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="L-410"><a href="#L-410"><span class="linenos">410</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
|
||||
</span><span id="L-411"><a href="#L-411"><span class="linenos">411</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="L-412"><a href="#L-412"><span class="linenos">412</span></a> <span class="n">keys</span> <span class="o">=</span> <span class="n">keys</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="L-413"><a href="#L-413"><span class="linenos">413</span></a>
|
||||
</span><span id="L-414"><a href="#L-414"><span class="linenos">414</span></a> <span class="k">return</span> <span class="n">tables</span>
|
||||
</span><span id="L-415"><a href="#L-415"><span class="linenos">415</span></a>
|
||||
</span><span id="L-416"><a href="#L-416"><span class="linenos">416</span></a>
|
||||
</span><span id="L-417"><a href="#L-417"><span class="linenos">417</span></a><span class="k">def</span> <span class="nf">nested_get</span><span class="p">(</span>
|
||||
</span><span id="L-418"><a href="#L-418"><span class="linenos">418</span></a> <span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="o">*</span><span class="n">path</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">],</span> <span class="n">raise_on_missing</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="L-419"><a href="#L-419"><span class="linenos">419</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]:</span>
|
||||
</span><span id="L-420"><a href="#L-420"><span class="linenos">420</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-421"><a href="#L-421"><span class="linenos">421</span></a><span class="sd"> Get a value for a nested dictionary.</span>
|
||||
</span><span id="L-414"><a href="#L-414"><span class="linenos">414</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">schema</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="L-415"><a href="#L-415"><span class="linenos">415</span></a> <span class="k">if</span> <span class="n">depth</span> <span class="o">>=</span> <span class="mi">2</span><span class="p">:</span>
|
||||
</span><span id="L-416"><a href="#L-416"><span class="linenos">416</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">flatten_schema</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">depth</span> <span class="o">-</span> <span class="mi">1</span><span class="p">,</span> <span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">]))</span>
|
||||
</span><span id="L-417"><a href="#L-417"><span class="linenos">417</span></a> <span class="k">elif</span> <span class="n">depth</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-418"><a href="#L-418"><span class="linenos">418</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">])</span>
|
||||
</span><span id="L-419"><a href="#L-419"><span class="linenos">419</span></a>
|
||||
</span><span id="L-420"><a href="#L-420"><span class="linenos">420</span></a> <span class="k">return</span> <span class="n">tables</span>
|
||||
</span><span id="L-421"><a href="#L-421"><span class="linenos">421</span></a>
|
||||
</span><span id="L-422"><a href="#L-422"><span class="linenos">422</span></a>
|
||||
</span><span id="L-423"><a href="#L-423"><span class="linenos">423</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-424"><a href="#L-424"><span class="linenos">424</span></a><span class="sd"> d: the dictionary to search.</span>
|
||||
</span><span id="L-425"><a href="#L-425"><span class="linenos">425</span></a><span class="sd"> *path: tuples of (name, key), where:</span>
|
||||
</span><span id="L-426"><a href="#L-426"><span class="linenos">426</span></a><span class="sd"> `key` is the key in the dictionary to get.</span>
|
||||
</span><span id="L-427"><a href="#L-427"><span class="linenos">427</span></a><span class="sd"> `name` is a string to use in the error if `key` isn't found.</span>
|
||||
</span><span id="L-423"><a href="#L-423"><span class="linenos">423</span></a><span class="k">def</span> <span class="nf">nested_get</span><span class="p">(</span>
|
||||
</span><span id="L-424"><a href="#L-424"><span class="linenos">424</span></a> <span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="o">*</span><span class="n">path</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">],</span> <span class="n">raise_on_missing</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="L-425"><a href="#L-425"><span class="linenos">425</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]:</span>
|
||||
</span><span id="L-426"><a href="#L-426"><span class="linenos">426</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-427"><a href="#L-427"><span class="linenos">427</span></a><span class="sd"> Get a value for a nested dictionary.</span>
|
||||
</span><span id="L-428"><a href="#L-428"><span class="linenos">428</span></a>
|
||||
</span><span id="L-429"><a href="#L-429"><span class="linenos">429</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-430"><a href="#L-430"><span class="linenos">430</span></a><span class="sd"> The value or None if it doesn't exist.</span>
|
||||
</span><span id="L-431"><a href="#L-431"><span class="linenos">431</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-432"><a href="#L-432"><span class="linenos">432</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">path</span><span class="p">:</span>
|
||||
</span><span id="L-433"><a href="#L-433"><span class="linenos">433</span></a> <span class="n">d</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="L-434"><a href="#L-434"><span class="linenos">434</span></a> <span class="k">if</span> <span class="n">d</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-435"><a href="#L-435"><span class="linenos">435</span></a> <span class="k">if</span> <span class="n">raise_on_missing</span><span class="p">:</span>
|
||||
</span><span id="L-436"><a href="#L-436"><span class="linenos">436</span></a> <span class="n">name</span> <span class="o">=</span> <span class="s2">"table"</span> <span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="s2">"this"</span> <span class="k">else</span> <span class="n">name</span>
|
||||
</span><span id="L-437"><a href="#L-437"><span class="linenos">437</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">: </span><span class="si">{</span><span class="n">key</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="L-438"><a href="#L-438"><span class="linenos">438</span></a> <span class="k">return</span> <span class="kc">None</span>
|
||||
</span><span id="L-439"><a href="#L-439"><span class="linenos">439</span></a>
|
||||
</span><span id="L-440"><a href="#L-440"><span class="linenos">440</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="L-441"><a href="#L-441"><span class="linenos">441</span></a>
|
||||
</span><span id="L-442"><a href="#L-442"><span class="linenos">442</span></a>
|
||||
</span><span id="L-443"><a href="#L-443"><span class="linenos">443</span></a><span class="k">def</span> <span class="nf">nested_set</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Sequence</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="L-444"><a href="#L-444"><span class="linenos">444</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-445"><a href="#L-445"><span class="linenos">445</span></a><span class="sd"> In-place set a value for a nested dictionary</span>
|
||||
</span><span id="L-446"><a href="#L-446"><span class="linenos">446</span></a>
|
||||
</span><span id="L-447"><a href="#L-447"><span class="linenos">447</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="L-448"><a href="#L-448"><span class="linenos">448</span></a><span class="sd"> >>> nested_set({}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="L-449"><a href="#L-449"><span class="linenos">449</span></a><span class="sd"> {'top_key': {'second_key': 'value'}}</span>
|
||||
</span><span id="L-450"><a href="#L-450"><span class="linenos">450</span></a>
|
||||
</span><span id="L-451"><a href="#L-451"><span class="linenos">451</span></a><span class="sd"> >>> nested_set({"top_key": {"third_key": "third_value"}}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="L-452"><a href="#L-452"><span class="linenos">452</span></a><span class="sd"> {'top_key': {'third_key': 'third_value', 'second_key': 'value'}}</span>
|
||||
</span><span id="L-453"><a href="#L-453"><span class="linenos">453</span></a>
|
||||
</span><span id="L-454"><a href="#L-454"><span class="linenos">454</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-455"><a href="#L-455"><span class="linenos">455</span></a><span class="sd"> d: dictionary to update.</span>
|
||||
</span><span id="L-456"><a href="#L-456"><span class="linenos">456</span></a><span class="sd"> keys: the keys that makeup the path to `value`.</span>
|
||||
</span><span id="L-457"><a href="#L-457"><span class="linenos">457</span></a><span class="sd"> value: the value to set in the dictionary for the given key path.</span>
|
||||
</span><span id="L-458"><a href="#L-458"><span class="linenos">458</span></a>
|
||||
</span><span id="L-459"><a href="#L-459"><span class="linenos">459</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-460"><a href="#L-460"><span class="linenos">460</span></a><span class="sd"> The (possibly) updated dictionary.</span>
|
||||
</span><span id="L-461"><a href="#L-461"><span class="linenos">461</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-462"><a href="#L-462"><span class="linenos">462</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">keys</span><span class="p">:</span>
|
||||
</span><span id="L-463"><a href="#L-463"><span class="linenos">463</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="L-429"><a href="#L-429"><span class="linenos">429</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-430"><a href="#L-430"><span class="linenos">430</span></a><span class="sd"> d: the dictionary to search.</span>
|
||||
</span><span id="L-431"><a href="#L-431"><span class="linenos">431</span></a><span class="sd"> *path: tuples of (name, key), where:</span>
|
||||
</span><span id="L-432"><a href="#L-432"><span class="linenos">432</span></a><span class="sd"> `key` is the key in the dictionary to get.</span>
|
||||
</span><span id="L-433"><a href="#L-433"><span class="linenos">433</span></a><span class="sd"> `name` is a string to use in the error if `key` isn't found.</span>
|
||||
</span><span id="L-434"><a href="#L-434"><span class="linenos">434</span></a>
|
||||
</span><span id="L-435"><a href="#L-435"><span class="linenos">435</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-436"><a href="#L-436"><span class="linenos">436</span></a><span class="sd"> The value or None if it doesn't exist.</span>
|
||||
</span><span id="L-437"><a href="#L-437"><span class="linenos">437</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-438"><a href="#L-438"><span class="linenos">438</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">path</span><span class="p">:</span>
|
||||
</span><span id="L-439"><a href="#L-439"><span class="linenos">439</span></a> <span class="n">d</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="L-440"><a href="#L-440"><span class="linenos">440</span></a> <span class="k">if</span> <span class="n">d</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="L-441"><a href="#L-441"><span class="linenos">441</span></a> <span class="k">if</span> <span class="n">raise_on_missing</span><span class="p">:</span>
|
||||
</span><span id="L-442"><a href="#L-442"><span class="linenos">442</span></a> <span class="n">name</span> <span class="o">=</span> <span class="s2">"table"</span> <span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="s2">"this"</span> <span class="k">else</span> <span class="n">name</span>
|
||||
</span><span id="L-443"><a href="#L-443"><span class="linenos">443</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">: </span><span class="si">{</span><span class="n">key</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="L-444"><a href="#L-444"><span class="linenos">444</span></a> <span class="k">return</span> <span class="kc">None</span>
|
||||
</span><span id="L-445"><a href="#L-445"><span class="linenos">445</span></a>
|
||||
</span><span id="L-446"><a href="#L-446"><span class="linenos">446</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="L-447"><a href="#L-447"><span class="linenos">447</span></a>
|
||||
</span><span id="L-448"><a href="#L-448"><span class="linenos">448</span></a>
|
||||
</span><span id="L-449"><a href="#L-449"><span class="linenos">449</span></a><span class="k">def</span> <span class="nf">nested_set</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Sequence</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="L-450"><a href="#L-450"><span class="linenos">450</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="L-451"><a href="#L-451"><span class="linenos">451</span></a><span class="sd"> In-place set a value for a nested dictionary</span>
|
||||
</span><span id="L-452"><a href="#L-452"><span class="linenos">452</span></a>
|
||||
</span><span id="L-453"><a href="#L-453"><span class="linenos">453</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="L-454"><a href="#L-454"><span class="linenos">454</span></a><span class="sd"> >>> nested_set({}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="L-455"><a href="#L-455"><span class="linenos">455</span></a><span class="sd"> {'top_key': {'second_key': 'value'}}</span>
|
||||
</span><span id="L-456"><a href="#L-456"><span class="linenos">456</span></a>
|
||||
</span><span id="L-457"><a href="#L-457"><span class="linenos">457</span></a><span class="sd"> >>> nested_set({"top_key": {"third_key": "third_value"}}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="L-458"><a href="#L-458"><span class="linenos">458</span></a><span class="sd"> {'top_key': {'third_key': 'third_value', 'second_key': 'value'}}</span>
|
||||
</span><span id="L-459"><a href="#L-459"><span class="linenos">459</span></a>
|
||||
</span><span id="L-460"><a href="#L-460"><span class="linenos">460</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="L-461"><a href="#L-461"><span class="linenos">461</span></a><span class="sd"> d: dictionary to update.</span>
|
||||
</span><span id="L-462"><a href="#L-462"><span class="linenos">462</span></a><span class="sd"> keys: the keys that makeup the path to `value`.</span>
|
||||
</span><span id="L-463"><a href="#L-463"><span class="linenos">463</span></a><span class="sd"> value: the value to set in the dictionary for the given key path.</span>
|
||||
</span><span id="L-464"><a href="#L-464"><span class="linenos">464</span></a>
|
||||
</span><span id="L-465"><a href="#L-465"><span class="linenos">465</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">keys</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-466"><a href="#L-466"><span class="linenos">466</span></a> <span class="n">d</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="mi">0</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="L-467"><a href="#L-467"><span class="linenos">467</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="L-468"><a href="#L-468"><span class="linenos">468</span></a>
|
||||
</span><span id="L-469"><a href="#L-469"><span class="linenos">469</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">d</span>
|
||||
</span><span id="L-470"><a href="#L-470"><span class="linenos">470</span></a> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">keys</span><span class="p">[:</span><span class="o">-</span><span class="mi">1</span><span class="p">]:</span>
|
||||
</span><span id="L-471"><a href="#L-471"><span class="linenos">471</span></a> <span class="k">if</span> <span class="n">key</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">subd</span><span class="p">:</span>
|
||||
</span><span id="L-472"><a href="#L-472"><span class="linenos">472</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="o">.</span><span class="n">setdefault</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="p">{})</span>
|
||||
</span><span id="L-473"><a href="#L-473"><span class="linenos">473</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-474"><a href="#L-474"><span class="linenos">474</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>
|
||||
</span><span id="L-475"><a href="#L-475"><span class="linenos">475</span></a>
|
||||
</span><span id="L-476"><a href="#L-476"><span class="linenos">476</span></a> <span class="n">subd</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="L-477"><a href="#L-477"><span class="linenos">477</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="L-465"><a href="#L-465"><span class="linenos">465</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="L-466"><a href="#L-466"><span class="linenos">466</span></a><span class="sd"> The (possibly) updated dictionary.</span>
|
||||
</span><span id="L-467"><a href="#L-467"><span class="linenos">467</span></a><span class="sd"> """</span>
|
||||
</span><span id="L-468"><a href="#L-468"><span class="linenos">468</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">keys</span><span class="p">:</span>
|
||||
</span><span id="L-469"><a href="#L-469"><span class="linenos">469</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="L-470"><a href="#L-470"><span class="linenos">470</span></a>
|
||||
</span><span id="L-471"><a href="#L-471"><span class="linenos">471</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">keys</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="L-472"><a href="#L-472"><span class="linenos">472</span></a> <span class="n">d</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="mi">0</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="L-473"><a href="#L-473"><span class="linenos">473</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="L-474"><a href="#L-474"><span class="linenos">474</span></a>
|
||||
</span><span id="L-475"><a href="#L-475"><span class="linenos">475</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">d</span>
|
||||
</span><span id="L-476"><a href="#L-476"><span class="linenos">476</span></a> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">keys</span><span class="p">[:</span><span class="o">-</span><span class="mi">1</span><span class="p">]:</span>
|
||||
</span><span id="L-477"><a href="#L-477"><span class="linenos">477</span></a> <span class="k">if</span> <span class="n">key</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">subd</span><span class="p">:</span>
|
||||
</span><span id="L-478"><a href="#L-478"><span class="linenos">478</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="o">.</span><span class="n">setdefault</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="p">{})</span>
|
||||
</span><span id="L-479"><a href="#L-479"><span class="linenos">479</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="L-480"><a href="#L-480"><span class="linenos">480</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>
|
||||
</span><span id="L-481"><a href="#L-481"><span class="linenos">481</span></a>
|
||||
</span><span id="L-482"><a href="#L-482"><span class="linenos">482</span></a> <span class="n">subd</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="L-483"><a href="#L-483"><span class="linenos">483</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1225,7 +1231,7 @@ For example, a generic mapping type might be defined as::</p>
|
|||
</span><span id="MappingSchema-290"><a href="#MappingSchema-290"><span class="linenos">290</span></a>
|
||||
</span><span id="MappingSchema-291"><a href="#MappingSchema-291"><span class="linenos">291</span></a> <span class="k">def</span> <span class="nf">_normalize</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-292"><a href="#MappingSchema-292"><span class="linenos">292</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="MappingSchema-293"><a href="#MappingSchema-293"><span class="linenos">293</span></a><span class="sd"> Converts all identifiers in the schema into lowercase, unless they're quoted.</span>
|
||||
</span><span id="MappingSchema-293"><a href="#MappingSchema-293"><span class="linenos">293</span></a><span class="sd"> Normalizes all identifiers in the schema.</span>
|
||||
</span><span id="MappingSchema-294"><a href="#MappingSchema-294"><span class="linenos">294</span></a>
|
||||
</span><span id="MappingSchema-295"><a href="#MappingSchema-295"><span class="linenos">295</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="MappingSchema-296"><a href="#MappingSchema-296"><span class="linenos">296</span></a><span class="sd"> schema: the schema to normalize.</span>
|
||||
|
|
@ -1240,72 +1246,78 @@ For example, a generic mapping type might be defined as::</p>
|
|||
</span><span id="MappingSchema-305"><a href="#MappingSchema-305"><span class="linenos">305</span></a> <span class="n">columns</span> <span class="o">=</span> <span class="n">nested_get</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="o">*</span><span class="nb">zip</span><span class="p">(</span><span class="n">keys</span><span class="p">,</span> <span class="n">keys</span><span class="p">))</span>
|
||||
</span><span id="MappingSchema-306"><a href="#MappingSchema-306"><span class="linenos">306</span></a> <span class="k">assert</span> <span class="n">columns</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
|
||||
</span><span id="MappingSchema-307"><a href="#MappingSchema-307"><span class="linenos">307</span></a>
|
||||
</span><span id="MappingSchema-308"><a href="#MappingSchema-308"><span class="linenos">308</span></a> <span class="n">normalized_keys</span> <span class="o">=</span> <span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">_normalize_name</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">keys</span><span class="p">]</span>
|
||||
</span><span id="MappingSchema-309"><a href="#MappingSchema-309"><span class="linenos">309</span></a> <span class="k">for</span> <span class="n">column_name</span><span class="p">,</span> <span class="n">column_type</span> <span class="ow">in</span> <span class="n">columns</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="MappingSchema-310"><a href="#MappingSchema-310"><span class="linenos">310</span></a> <span class="n">nested_set</span><span class="p">(</span>
|
||||
</span><span id="MappingSchema-311"><a href="#MappingSchema-311"><span class="linenos">311</span></a> <span class="n">normalized_mapping</span><span class="p">,</span>
|
||||
</span><span id="MappingSchema-312"><a href="#MappingSchema-312"><span class="linenos">312</span></a> <span class="n">normalized_keys</span> <span class="o">+</span> <span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">_normalize_name</span><span class="p">(</span><span class="n">column_name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)],</span>
|
||||
</span><span id="MappingSchema-313"><a href="#MappingSchema-313"><span class="linenos">313</span></a> <span class="n">column_type</span><span class="p">,</span>
|
||||
</span><span id="MappingSchema-314"><a href="#MappingSchema-314"><span class="linenos">314</span></a> <span class="p">)</span>
|
||||
</span><span id="MappingSchema-315"><a href="#MappingSchema-315"><span class="linenos">315</span></a>
|
||||
</span><span id="MappingSchema-316"><a href="#MappingSchema-316"><span class="linenos">316</span></a> <span class="k">return</span> <span class="n">normalized_mapping</span>
|
||||
</span><span id="MappingSchema-308"><a href="#MappingSchema-308"><span class="linenos">308</span></a> <span class="n">normalized_keys</span> <span class="o">=</span> <span class="p">[</span>
|
||||
</span><span id="MappingSchema-309"><a href="#MappingSchema-309"><span class="linenos">309</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_normalize_name</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">,</span> <span class="n">is_table</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">keys</span>
|
||||
</span><span id="MappingSchema-310"><a href="#MappingSchema-310"><span class="linenos">310</span></a> <span class="p">]</span>
|
||||
</span><span id="MappingSchema-311"><a href="#MappingSchema-311"><span class="linenos">311</span></a> <span class="k">for</span> <span class="n">column_name</span><span class="p">,</span> <span class="n">column_type</span> <span class="ow">in</span> <span class="n">columns</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="MappingSchema-312"><a href="#MappingSchema-312"><span class="linenos">312</span></a> <span class="n">nested_set</span><span class="p">(</span>
|
||||
</span><span id="MappingSchema-313"><a href="#MappingSchema-313"><span class="linenos">313</span></a> <span class="n">normalized_mapping</span><span class="p">,</span>
|
||||
</span><span id="MappingSchema-314"><a href="#MappingSchema-314"><span class="linenos">314</span></a> <span class="n">normalized_keys</span> <span class="o">+</span> <span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">_normalize_name</span><span class="p">(</span><span class="n">column_name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)],</span>
|
||||
</span><span id="MappingSchema-315"><a href="#MappingSchema-315"><span class="linenos">315</span></a> <span class="n">column_type</span><span class="p">,</span>
|
||||
</span><span id="MappingSchema-316"><a href="#MappingSchema-316"><span class="linenos">316</span></a> <span class="p">)</span>
|
||||
</span><span id="MappingSchema-317"><a href="#MappingSchema-317"><span class="linenos">317</span></a>
|
||||
</span><span id="MappingSchema-318"><a href="#MappingSchema-318"><span class="linenos">318</span></a> <span class="k">def</span> <span class="nf">_normalize_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-319"><a href="#MappingSchema-319"><span class="linenos">319</span></a> <span class="n">normalized_table</span> <span class="o">=</span> <span class="n">table</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
</span><span id="MappingSchema-320"><a href="#MappingSchema-320"><span class="linenos">320</span></a>
|
||||
</span><span id="MappingSchema-321"><a href="#MappingSchema-321"><span class="linenos">321</span></a> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="n">TABLE_ARGS</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-322"><a href="#MappingSchema-322"><span class="linenos">322</span></a> <span class="n">value</span> <span class="o">=</span> <span class="n">normalized_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-323"><a href="#MappingSchema-323"><span class="linenos">323</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="p">(</span><span class="nb">str</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)):</span>
|
||||
</span><span id="MappingSchema-324"><a href="#MappingSchema-324"><span class="linenos">324</span></a> <span class="n">normalized_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="MappingSchema-325"><a href="#MappingSchema-325"><span class="linenos">325</span></a> <span class="n">arg</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_normalize_name</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">))</span>
|
||||
</span><span id="MappingSchema-326"><a href="#MappingSchema-326"><span class="linenos">326</span></a> <span class="p">)</span>
|
||||
</span><span id="MappingSchema-327"><a href="#MappingSchema-327"><span class="linenos">327</span></a>
|
||||
</span><span id="MappingSchema-328"><a href="#MappingSchema-328"><span class="linenos">328</span></a> <span class="k">return</span> <span class="n">normalized_table</span>
|
||||
</span><span id="MappingSchema-329"><a href="#MappingSchema-329"><span class="linenos">329</span></a>
|
||||
</span><span id="MappingSchema-330"><a href="#MappingSchema-330"><span class="linenos">330</span></a> <span class="k">def</span> <span class="nf">_normalize_name</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-331"><a href="#MappingSchema-331"><span class="linenos">331</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span>
|
||||
</span><span id="MappingSchema-318"><a href="#MappingSchema-318"><span class="linenos">318</span></a> <span class="k">return</span> <span class="n">normalized_mapping</span>
|
||||
</span><span id="MappingSchema-319"><a href="#MappingSchema-319"><span class="linenos">319</span></a>
|
||||
</span><span id="MappingSchema-320"><a href="#MappingSchema-320"><span class="linenos">320</span></a> <span class="k">def</span> <span class="nf">_normalize_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-321"><a href="#MappingSchema-321"><span class="linenos">321</span></a> <span class="n">normalized_table</span> <span class="o">=</span> <span class="n">table</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
|
||||
</span><span id="MappingSchema-322"><a href="#MappingSchema-322"><span class="linenos">322</span></a>
|
||||
</span><span id="MappingSchema-323"><a href="#MappingSchema-323"><span class="linenos">323</span></a> <span class="k">for</span> <span class="n">arg</span> <span class="ow">in</span> <span class="n">TABLE_ARGS</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-324"><a href="#MappingSchema-324"><span class="linenos">324</span></a> <span class="n">value</span> <span class="o">=</span> <span class="n">normalized_table</span><span class="o">.</span><span class="n">args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">arg</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-325"><a href="#MappingSchema-325"><span class="linenos">325</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="p">(</span><span class="nb">str</span><span class="p">,</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)):</span>
|
||||
</span><span id="MappingSchema-326"><a href="#MappingSchema-326"><span class="linenos">326</span></a> <span class="n">normalized_table</span><span class="o">.</span><span class="n">set</span><span class="p">(</span>
|
||||
</span><span id="MappingSchema-327"><a href="#MappingSchema-327"><span class="linenos">327</span></a> <span class="n">arg</span><span class="p">,</span>
|
||||
</span><span id="MappingSchema-328"><a href="#MappingSchema-328"><span class="linenos">328</span></a> <span class="n">exp</span><span class="o">.</span><span class="n">to_identifier</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_normalize_name</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">is_table</span><span class="o">=</span><span class="kc">True</span><span class="p">)),</span>
|
||||
</span><span id="MappingSchema-329"><a href="#MappingSchema-329"><span class="linenos">329</span></a> <span class="p">)</span>
|
||||
</span><span id="MappingSchema-330"><a href="#MappingSchema-330"><span class="linenos">330</span></a>
|
||||
</span><span id="MappingSchema-331"><a href="#MappingSchema-331"><span class="linenos">331</span></a> <span class="k">return</span> <span class="n">normalized_table</span>
|
||||
</span><span id="MappingSchema-332"><a href="#MappingSchema-332"><span class="linenos">332</span></a>
|
||||
</span><span id="MappingSchema-333"><a href="#MappingSchema-333"><span class="linenos">333</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-334"><a href="#MappingSchema-334"><span class="linenos">334</span></a> <span class="n">identifier</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">maybe_parse</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-335"><a href="#MappingSchema-335"><span class="linenos">335</span></a> <span class="k">except</span> <span class="n">ParseError</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-336"><a href="#MappingSchema-336"><span class="linenos">336</span></a> <span class="k">return</span> <span class="n">name</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">name</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="MappingSchema-333"><a href="#MappingSchema-333"><span class="linenos">333</span></a> <span class="k">def</span> <span class="nf">_normalize_name</span><span class="p">(</span>
|
||||
</span><span id="MappingSchema-334"><a href="#MappingSchema-334"><span class="linenos">334</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">:</span> <span class="nb">str</span> <span class="o">|</span> <span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span> <span class="n">is_table</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span>
|
||||
</span><span id="MappingSchema-335"><a href="#MappingSchema-335"><span class="linenos">335</span></a> <span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-336"><a href="#MappingSchema-336"><span class="linenos">336</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span>
|
||||
</span><span id="MappingSchema-337"><a href="#MappingSchema-337"><span class="linenos">337</span></a>
|
||||
</span><span id="MappingSchema-338"><a href="#MappingSchema-338"><span class="linenos">338</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">identifier</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="MappingSchema-339"><a href="#MappingSchema-339"><span class="linenos">339</span></a>
|
||||
</span><span id="MappingSchema-340"><a href="#MappingSchema-340"><span class="linenos">340</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">normalize</span> <span class="ow">or</span> <span class="n">identifier</span><span class="o">.</span><span class="n">quoted</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-341"><a href="#MappingSchema-341"><span class="linenos">341</span></a> <span class="k">return</span> <span class="n">name</span>
|
||||
</span><span id="MappingSchema-338"><a href="#MappingSchema-338"><span class="linenos">338</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-339"><a href="#MappingSchema-339"><span class="linenos">339</span></a> <span class="n">identifier</span> <span class="o">=</span> <span class="n">sqlglot</span><span class="o">.</span><span class="n">maybe_parse</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Identifier</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-340"><a href="#MappingSchema-340"><span class="linenos">340</span></a> <span class="k">except</span> <span class="n">ParseError</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-341"><a href="#MappingSchema-341"><span class="linenos">341</span></a> <span class="k">return</span> <span class="n">name</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">name</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="MappingSchema-342"><a href="#MappingSchema-342"><span class="linenos">342</span></a>
|
||||
</span><span id="MappingSchema-343"><a href="#MappingSchema-343"><span class="linenos">343</span></a> <span class="k">return</span> <span class="n">name</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span> <span class="k">if</span> <span class="n">dialect</span> <span class="ow">in</span> <span class="n">RESOLVES_IDENTIFIERS_AS_UPPERCASE</span> <span class="k">else</span> <span class="n">name</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span>
|
||||
</span><span id="MappingSchema-344"><a href="#MappingSchema-344"><span class="linenos">344</span></a>
|
||||
</span><span id="MappingSchema-345"><a href="#MappingSchema-345"><span class="linenos">345</span></a> <span class="k">def</span> <span class="nf">_depth</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">int</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-346"><a href="#MappingSchema-346"><span class="linenos">346</span></a> <span class="c1"># The columns themselves are a mapping, but we don't want to include those</span>
|
||||
</span><span id="MappingSchema-347"><a href="#MappingSchema-347"><span class="linenos">347</span></a> <span class="k">return</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">_depth</span><span class="p">()</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
</span><span id="MappingSchema-348"><a href="#MappingSchema-348"><span class="linenos">348</span></a>
|
||||
</span><span id="MappingSchema-349"><a href="#MappingSchema-349"><span class="linenos">349</span></a> <span class="k">def</span> <span class="nf">_ensure_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span> <span class="o">|</span> <span class="nb">str</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-350"><a href="#MappingSchema-350"><span class="linenos">350</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">maybe_parse</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-351"><a href="#MappingSchema-351"><span class="linenos">351</span></a>
|
||||
</span><span id="MappingSchema-352"><a href="#MappingSchema-352"><span class="linenos">352</span></a> <span class="k">def</span> <span class="nf">_to_data_type</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">schema_type</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-353"><a href="#MappingSchema-353"><span class="linenos">353</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="MappingSchema-354"><a href="#MappingSchema-354"><span class="linenos">354</span></a><span class="sd"> Convert a type represented as a string to the corresponding `sqlglot.exp.DataType` object.</span>
|
||||
</span><span id="MappingSchema-355"><a href="#MappingSchema-355"><span class="linenos">355</span></a>
|
||||
</span><span id="MappingSchema-356"><a href="#MappingSchema-356"><span class="linenos">356</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="MappingSchema-357"><a href="#MappingSchema-357"><span class="linenos">357</span></a><span class="sd"> schema_type: the type we want to convert.</span>
|
||||
</span><span id="MappingSchema-358"><a href="#MappingSchema-358"><span class="linenos">358</span></a><span class="sd"> dialect: the SQL dialect that will be used to parse `schema_type`, if needed.</span>
|
||||
</span><span id="MappingSchema-359"><a href="#MappingSchema-359"><span class="linenos">359</span></a>
|
||||
</span><span id="MappingSchema-360"><a href="#MappingSchema-360"><span class="linenos">360</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="MappingSchema-361"><a href="#MappingSchema-361"><span class="linenos">361</span></a><span class="sd"> The resulting expression type.</span>
|
||||
</span><span id="MappingSchema-362"><a href="#MappingSchema-362"><span class="linenos">362</span></a><span class="sd"> """</span>
|
||||
</span><span id="MappingSchema-363"><a href="#MappingSchema-363"><span class="linenos">363</span></a> <span class="k">if</span> <span class="n">schema_type</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-364"><a href="#MappingSchema-364"><span class="linenos">364</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span>
|
||||
</span><span id="MappingSchema-343"><a href="#MappingSchema-343"><span class="linenos">343</span></a> <span class="n">name</span> <span class="o">=</span> <span class="n">identifier</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="MappingSchema-344"><a href="#MappingSchema-344"><span class="linenos">344</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">normalize</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-345"><a href="#MappingSchema-345"><span class="linenos">345</span></a> <span class="k">return</span> <span class="n">name</span>
|
||||
</span><span id="MappingSchema-346"><a href="#MappingSchema-346"><span class="linenos">346</span></a>
|
||||
</span><span id="MappingSchema-347"><a href="#MappingSchema-347"><span class="linenos">347</span></a> <span class="c1"># This can be useful for normalize_identifier</span>
|
||||
</span><span id="MappingSchema-348"><a href="#MappingSchema-348"><span class="linenos">348</span></a> <span class="n">identifier</span><span class="o">.</span><span class="n">meta</span><span class="p">[</span><span class="s2">"is_table"</span><span class="p">]</span> <span class="o">=</span> <span class="n">is_table</span>
|
||||
</span><span id="MappingSchema-349"><a href="#MappingSchema-349"><span class="linenos">349</span></a> <span class="k">return</span> <span class="n">Dialect</span><span class="o">.</span><span class="n">get_or_raise</span><span class="p">(</span><span class="n">dialect</span><span class="p">)</span><span class="o">.</span><span class="n">normalize_identifier</span><span class="p">(</span><span class="n">identifier</span><span class="p">)</span><span class="o">.</span><span class="n">name</span>
|
||||
</span><span id="MappingSchema-350"><a href="#MappingSchema-350"><span class="linenos">350</span></a>
|
||||
</span><span id="MappingSchema-351"><a href="#MappingSchema-351"><span class="linenos">351</span></a> <span class="k">def</span> <span class="nf">_depth</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-></span> <span class="nb">int</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-352"><a href="#MappingSchema-352"><span class="linenos">352</span></a> <span class="c1"># The columns themselves are a mapping, but we don't want to include those</span>
|
||||
</span><span id="MappingSchema-353"><a href="#MappingSchema-353"><span class="linenos">353</span></a> <span class="k">return</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">_depth</span><span class="p">()</span> <span class="o">-</span> <span class="mi">1</span>
|
||||
</span><span id="MappingSchema-354"><a href="#MappingSchema-354"><span class="linenos">354</span></a>
|
||||
</span><span id="MappingSchema-355"><a href="#MappingSchema-355"><span class="linenos">355</span></a> <span class="k">def</span> <span class="nf">_ensure_table</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">table</span><span class="p">:</span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span> <span class="o">|</span> <span class="nb">str</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-356"><a href="#MappingSchema-356"><span class="linenos">356</span></a> <span class="k">return</span> <span class="n">exp</span><span class="o">.</span><span class="n">maybe_parse</span><span class="p">(</span><span class="n">table</span><span class="p">,</span> <span class="n">into</span><span class="o">=</span><span class="n">exp</span><span class="o">.</span><span class="n">Table</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-357"><a href="#MappingSchema-357"><span class="linenos">357</span></a>
|
||||
</span><span id="MappingSchema-358"><a href="#MappingSchema-358"><span class="linenos">358</span></a> <span class="k">def</span> <span class="nf">_to_data_type</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">schema_type</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">dialect</span><span class="p">:</span> <span class="n">DialectType</span> <span class="o">=</span> <span class="kc">None</span><span class="p">)</span> <span class="o">-></span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-359"><a href="#MappingSchema-359"><span class="linenos">359</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="MappingSchema-360"><a href="#MappingSchema-360"><span class="linenos">360</span></a><span class="sd"> Convert a type represented as a string to the corresponding `sqlglot.exp.DataType` object.</span>
|
||||
</span><span id="MappingSchema-361"><a href="#MappingSchema-361"><span class="linenos">361</span></a>
|
||||
</span><span id="MappingSchema-362"><a href="#MappingSchema-362"><span class="linenos">362</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="MappingSchema-363"><a href="#MappingSchema-363"><span class="linenos">363</span></a><span class="sd"> schema_type: the type we want to convert.</span>
|
||||
</span><span id="MappingSchema-364"><a href="#MappingSchema-364"><span class="linenos">364</span></a><span class="sd"> dialect: the SQL dialect that will be used to parse `schema_type`, if needed.</span>
|
||||
</span><span id="MappingSchema-365"><a href="#MappingSchema-365"><span class="linenos">365</span></a>
|
||||
</span><span id="MappingSchema-366"><a href="#MappingSchema-366"><span class="linenos">366</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-367"><a href="#MappingSchema-367"><span class="linenos">367</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">build</span><span class="p">(</span><span class="n">schema_type</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-368"><a href="#MappingSchema-368"><span class="linenos">368</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span> <span class="o">=</span> <span class="n">expression</span>
|
||||
</span><span id="MappingSchema-369"><a href="#MappingSchema-369"><span class="linenos">369</span></a> <span class="k">except</span> <span class="ne">AttributeError</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-370"><a href="#MappingSchema-370"><span class="linenos">370</span></a> <span class="n">in_dialect</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">" in dialect </span><span class="si">{</span><span class="n">dialect</span><span class="si">}</span><span class="s2">"</span> <span class="k">if</span> <span class="n">dialect</span> <span class="k">else</span> <span class="s2">""</span>
|
||||
</span><span id="MappingSchema-371"><a href="#MappingSchema-371"><span class="linenos">371</span></a> <span class="k">raise</span> <span class="n">SchemaError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Failed to build type '</span><span class="si">{</span><span class="n">schema_type</span><span class="si">}</span><span class="s2">'</span><span class="si">{</span><span class="n">in_dialect</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-372"><a href="#MappingSchema-372"><span class="linenos">372</span></a>
|
||||
</span><span id="MappingSchema-373"><a href="#MappingSchema-373"><span class="linenos">373</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span>
|
||||
</span><span id="MappingSchema-366"><a href="#MappingSchema-366"><span class="linenos">366</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="MappingSchema-367"><a href="#MappingSchema-367"><span class="linenos">367</span></a><span class="sd"> The resulting expression type.</span>
|
||||
</span><span id="MappingSchema-368"><a href="#MappingSchema-368"><span class="linenos">368</span></a><span class="sd"> """</span>
|
||||
</span><span id="MappingSchema-369"><a href="#MappingSchema-369"><span class="linenos">369</span></a> <span class="k">if</span> <span class="n">schema_type</span> <span class="ow">not</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-370"><a href="#MappingSchema-370"><span class="linenos">370</span></a> <span class="n">dialect</span> <span class="o">=</span> <span class="n">dialect</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">dialect</span>
|
||||
</span><span id="MappingSchema-371"><a href="#MappingSchema-371"><span class="linenos">371</span></a>
|
||||
</span><span id="MappingSchema-372"><a href="#MappingSchema-372"><span class="linenos">372</span></a> <span class="k">try</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-373"><a href="#MappingSchema-373"><span class="linenos">373</span></a> <span class="n">expression</span> <span class="o">=</span> <span class="n">exp</span><span class="o">.</span><span class="n">DataType</span><span class="o">.</span><span class="n">build</span><span class="p">(</span><span class="n">schema_type</span><span class="p">,</span> <span class="n">dialect</span><span class="o">=</span><span class="n">dialect</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-374"><a href="#MappingSchema-374"><span class="linenos">374</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span> <span class="o">=</span> <span class="n">expression</span>
|
||||
</span><span id="MappingSchema-375"><a href="#MappingSchema-375"><span class="linenos">375</span></a> <span class="k">except</span> <span class="ne">AttributeError</span><span class="p">:</span>
|
||||
</span><span id="MappingSchema-376"><a href="#MappingSchema-376"><span class="linenos">376</span></a> <span class="n">in_dialect</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">" in dialect </span><span class="si">{</span><span class="n">dialect</span><span class="si">}</span><span class="s2">"</span> <span class="k">if</span> <span class="n">dialect</span> <span class="k">else</span> <span class="s2">""</span>
|
||||
</span><span id="MappingSchema-377"><a href="#MappingSchema-377"><span class="linenos">377</span></a> <span class="k">raise</span> <span class="n">SchemaError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Failed to build type '</span><span class="si">{</span><span class="n">schema_type</span><span class="si">}</span><span class="s2">'</span><span class="si">{</span><span class="n">in_dialect</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
|
||||
</span><span id="MappingSchema-378"><a href="#MappingSchema-378"><span class="linenos">378</span></a>
|
||||
</span><span id="MappingSchema-379"><a href="#MappingSchema-379"><span class="linenos">379</span></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_type_mapping_cache</span><span class="p">[</span><span class="n">schema_type</span><span class="p">]</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1604,11 +1616,11 @@ are assumed to be visible. The nesting should mirror that of the schema:
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#ensure_schema"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_schema-376"><a href="#ensure_schema-376"><span class="linenos">376</span></a><span class="k">def</span> <span class="nf">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">:</span> <span class="n">Schema</span> <span class="o">|</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">Schema</span><span class="p">:</span>
|
||||
</span><span id="ensure_schema-377"><a href="#ensure_schema-377"><span class="linenos">377</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">Schema</span><span class="p">):</span>
|
||||
</span><span id="ensure_schema-378"><a href="#ensure_schema-378"><span class="linenos">378</span></a> <span class="k">return</span> <span class="n">schema</span>
|
||||
</span><span id="ensure_schema-379"><a href="#ensure_schema-379"><span class="linenos">379</span></a>
|
||||
</span><span id="ensure_schema-380"><a href="#ensure_schema-380"><span class="linenos">380</span></a> <span class="k">return</span> <span class="n">MappingSchema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_schema-382"><a href="#ensure_schema-382"><span class="linenos">382</span></a><span class="k">def</span> <span class="nf">ensure_schema</span><span class="p">(</span><span class="n">schema</span><span class="p">:</span> <span class="n">Schema</span> <span class="o">|</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">],</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">Schema</span><span class="p">:</span>
|
||||
</span><span id="ensure_schema-383"><a href="#ensure_schema-383"><span class="linenos">383</span></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="n">Schema</span><span class="p">):</span>
|
||||
</span><span id="ensure_schema-384"><a href="#ensure_schema-384"><span class="linenos">384</span></a> <span class="k">return</span> <span class="n">schema</span>
|
||||
</span><span id="ensure_schema-385"><a href="#ensure_schema-385"><span class="linenos">385</span></a>
|
||||
</span><span id="ensure_schema-386"><a href="#ensure_schema-386"><span class="linenos">386</span></a> <span class="k">return</span> <span class="n">MappingSchema</span><span class="p">(</span><span class="n">schema</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1626,24 +1638,24 @@ are assumed to be visible. The nesting should mirror that of the schema:
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#ensure_column_mapping"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_column_mapping-383"><a href="#ensure_column_mapping-383"><span class="linenos">383</span></a><span class="k">def</span> <span class="nf">ensure_column_mapping</span><span class="p">(</span><span class="n">mapping</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">ColumnMapping</span><span class="p">])</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="ensure_column_mapping-384"><a href="#ensure_column_mapping-384"><span class="linenos">384</span></a> <span class="k">if</span> <span class="n">mapping</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="ensure_column_mapping-385"><a href="#ensure_column_mapping-385"><span class="linenos">385</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="ensure_column_mapping-386"><a href="#ensure_column_mapping-386"><span class="linenos">386</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">dict</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-387"><a href="#ensure_column_mapping-387"><span class="linenos">387</span></a> <span class="k">return</span> <span class="n">mapping</span>
|
||||
</span><span id="ensure_column_mapping-388"><a href="#ensure_column_mapping-388"><span class="linenos">388</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-389"><a href="#ensure_column_mapping-389"><span class="linenos">389</span></a> <span class="n">col_name_type_strs</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">","</span><span class="p">)]</span>
|
||||
</span><span id="ensure_column_mapping-390"><a href="#ensure_column_mapping-390"><span class="linenos">390</span></a> <span class="k">return</span> <span class="p">{</span>
|
||||
</span><span id="ensure_column_mapping-391"><a href="#ensure_column_mapping-391"><span class="linenos">391</span></a> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
|
||||
</span><span id="ensure_column_mapping-392"><a href="#ensure_column_mapping-392"><span class="linenos">392</span></a> <span class="k">for</span> <span class="n">name_type_str</span> <span class="ow">in</span> <span class="n">col_name_type_strs</span>
|
||||
</span><span id="ensure_column_mapping-393"><a href="#ensure_column_mapping-393"><span class="linenos">393</span></a> <span class="p">}</span>
|
||||
</span><span id="ensure_column_mapping-394"><a href="#ensure_column_mapping-394"><span class="linenos">394</span></a> <span class="c1"># Check if mapping looks like a DataFrame StructType</span>
|
||||
</span><span id="ensure_column_mapping-395"><a href="#ensure_column_mapping-395"><span class="linenos">395</span></a> <span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="s2">"simpleString"</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-396"><a href="#ensure_column_mapping-396"><span class="linenos">396</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">struct_field</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="n">struct_field</span><span class="o">.</span><span class="n">dataType</span><span class="o">.</span><span class="n">simpleString</span><span class="p">()</span> <span class="k">for</span> <span class="n">struct_field</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="ensure_column_mapping-397"><a href="#ensure_column_mapping-397"><span class="linenos">397</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-398"><a href="#ensure_column_mapping-398"><span class="linenos">398</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="ensure_column_mapping-399"><a href="#ensure_column_mapping-399"><span class="linenos">399</span></a>
|
||||
</span><span id="ensure_column_mapping-400"><a href="#ensure_column_mapping-400"><span class="linenos">400</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Invalid mapping provided: </span><span class="si">{</span><span class="nb">type</span><span class="p">(</span><span class="n">mapping</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="ensure_column_mapping-389"><a href="#ensure_column_mapping-389"><span class="linenos">389</span></a><span class="k">def</span> <span class="nf">ensure_column_mapping</span><span class="p">(</span><span class="n">mapping</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">ColumnMapping</span><span class="p">])</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="ensure_column_mapping-390"><a href="#ensure_column_mapping-390"><span class="linenos">390</span></a> <span class="k">if</span> <span class="n">mapping</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="ensure_column_mapping-391"><a href="#ensure_column_mapping-391"><span class="linenos">391</span></a> <span class="k">return</span> <span class="p">{}</span>
|
||||
</span><span id="ensure_column_mapping-392"><a href="#ensure_column_mapping-392"><span class="linenos">392</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">dict</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-393"><a href="#ensure_column_mapping-393"><span class="linenos">393</span></a> <span class="k">return</span> <span class="n">mapping</span>
|
||||
</span><span id="ensure_column_mapping-394"><a href="#ensure_column_mapping-394"><span class="linenos">394</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-395"><a href="#ensure_column_mapping-395"><span class="linenos">395</span></a> <span class="n">col_name_type_strs</span> <span class="o">=</span> <span class="p">[</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">","</span><span class="p">)]</span>
|
||||
</span><span id="ensure_column_mapping-396"><a href="#ensure_column_mapping-396"><span class="linenos">396</span></a> <span class="k">return</span> <span class="p">{</span>
|
||||
</span><span id="ensure_column_mapping-397"><a href="#ensure_column_mapping-397"><span class="linenos">397</span></a> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="n">name_type_str</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">":"</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
|
||||
</span><span id="ensure_column_mapping-398"><a href="#ensure_column_mapping-398"><span class="linenos">398</span></a> <span class="k">for</span> <span class="n">name_type_str</span> <span class="ow">in</span> <span class="n">col_name_type_strs</span>
|
||||
</span><span id="ensure_column_mapping-399"><a href="#ensure_column_mapping-399"><span class="linenos">399</span></a> <span class="p">}</span>
|
||||
</span><span id="ensure_column_mapping-400"><a href="#ensure_column_mapping-400"><span class="linenos">400</span></a> <span class="c1"># Check if mapping looks like a DataFrame StructType</span>
|
||||
</span><span id="ensure_column_mapping-401"><a href="#ensure_column_mapping-401"><span class="linenos">401</span></a> <span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="s2">"simpleString"</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-402"><a href="#ensure_column_mapping-402"><span class="linenos">402</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">struct_field</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="n">struct_field</span><span class="o">.</span><span class="n">dataType</span><span class="o">.</span><span class="n">simpleString</span><span class="p">()</span> <span class="k">for</span> <span class="n">struct_field</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="ensure_column_mapping-403"><a href="#ensure_column_mapping-403"><span class="linenos">403</span></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">mapping</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
|
||||
</span><span id="ensure_column_mapping-404"><a href="#ensure_column_mapping-404"><span class="linenos">404</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">x</span><span class="o">.</span><span class="n">strip</span><span class="p">():</span> <span class="kc">None</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mapping</span><span class="p">}</span>
|
||||
</span><span id="ensure_column_mapping-405"><a href="#ensure_column_mapping-405"><span class="linenos">405</span></a>
|
||||
</span><span id="ensure_column_mapping-406"><a href="#ensure_column_mapping-406"><span class="linenos">406</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Invalid mapping provided: </span><span class="si">{</span><span class="nb">type</span><span class="p">(</span><span class="n">mapping</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1661,19 +1673,19 @@ are assumed to be visible. The nesting should mirror that of the schema:
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#flatten_schema"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="flatten_schema-403"><a href="#flatten_schema-403"><span class="linenos">403</span></a><span class="k">def</span> <span class="nf">flatten_schema</span><span class="p">(</span>
|
||||
</span><span id="flatten_schema-404"><a href="#flatten_schema-404"><span class="linenos">404</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">depth</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="flatten_schema-405"><a href="#flatten_schema-405"><span class="linenos">405</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
|
||||
</span><span id="flatten_schema-406"><a href="#flatten_schema-406"><span class="linenos">406</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="flatten_schema-407"><a href="#flatten_schema-407"><span class="linenos">407</span></a> <span class="n">keys</span> <span class="o">=</span> <span class="n">keys</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="flatten_schema-408"><a href="#flatten_schema-408"><span class="linenos">408</span></a>
|
||||
</span><span id="flatten_schema-409"><a href="#flatten_schema-409"><span class="linenos">409</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">schema</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="flatten_schema-410"><a href="#flatten_schema-410"><span class="linenos">410</span></a> <span class="k">if</span> <span class="n">depth</span> <span class="o">>=</span> <span class="mi">2</span><span class="p">:</span>
|
||||
</span><span id="flatten_schema-411"><a href="#flatten_schema-411"><span class="linenos">411</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">flatten_schema</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">depth</span> <span class="o">-</span> <span class="mi">1</span><span class="p">,</span> <span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">]))</span>
|
||||
</span><span id="flatten_schema-412"><a href="#flatten_schema-412"><span class="linenos">412</span></a> <span class="k">elif</span> <span class="n">depth</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="flatten_schema-413"><a href="#flatten_schema-413"><span class="linenos">413</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">])</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="flatten_schema-409"><a href="#flatten_schema-409"><span class="linenos">409</span></a><span class="k">def</span> <span class="nf">flatten_schema</span><span class="p">(</span>
|
||||
</span><span id="flatten_schema-410"><a href="#flatten_schema-410"><span class="linenos">410</span></a> <span class="n">schema</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">depth</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span> <span class="o">=</span> <span class="kc">None</span>
|
||||
</span><span id="flatten_schema-411"><a href="#flatten_schema-411"><span class="linenos">411</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
|
||||
</span><span id="flatten_schema-412"><a href="#flatten_schema-412"><span class="linenos">412</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="p">[]</span>
|
||||
</span><span id="flatten_schema-413"><a href="#flatten_schema-413"><span class="linenos">413</span></a> <span class="n">keys</span> <span class="o">=</span> <span class="n">keys</span> <span class="ow">or</span> <span class="p">[]</span>
|
||||
</span><span id="flatten_schema-414"><a href="#flatten_schema-414"><span class="linenos">414</span></a>
|
||||
</span><span id="flatten_schema-415"><a href="#flatten_schema-415"><span class="linenos">415</span></a> <span class="k">return</span> <span class="n">tables</span>
|
||||
</span><span id="flatten_schema-415"><a href="#flatten_schema-415"><span class="linenos">415</span></a> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">schema</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
|
||||
</span><span id="flatten_schema-416"><a href="#flatten_schema-416"><span class="linenos">416</span></a> <span class="k">if</span> <span class="n">depth</span> <span class="o">>=</span> <span class="mi">2</span><span class="p">:</span>
|
||||
</span><span id="flatten_schema-417"><a href="#flatten_schema-417"><span class="linenos">417</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">flatten_schema</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">depth</span> <span class="o">-</span> <span class="mi">1</span><span class="p">,</span> <span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">]))</span>
|
||||
</span><span id="flatten_schema-418"><a href="#flatten_schema-418"><span class="linenos">418</span></a> <span class="k">elif</span> <span class="n">depth</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="flatten_schema-419"><a href="#flatten_schema-419"><span class="linenos">419</span></a> <span class="n">tables</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">keys</span> <span class="o">+</span> <span class="p">[</span><span class="n">k</span><span class="p">])</span>
|
||||
</span><span id="flatten_schema-420"><a href="#flatten_schema-420"><span class="linenos">420</span></a>
|
||||
</span><span id="flatten_schema-421"><a href="#flatten_schema-421"><span class="linenos">421</span></a> <span class="k">return</span> <span class="n">tables</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1691,30 +1703,30 @@ are assumed to be visible. The nesting should mirror that of the schema:
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#nested_get"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="nested_get-418"><a href="#nested_get-418"><span class="linenos">418</span></a><span class="k">def</span> <span class="nf">nested_get</span><span class="p">(</span>
|
||||
</span><span id="nested_get-419"><a href="#nested_get-419"><span class="linenos">419</span></a> <span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="o">*</span><span class="n">path</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">],</span> <span class="n">raise_on_missing</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="nested_get-420"><a href="#nested_get-420"><span class="linenos">420</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]:</span>
|
||||
</span><span id="nested_get-421"><a href="#nested_get-421"><span class="linenos">421</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="nested_get-422"><a href="#nested_get-422"><span class="linenos">422</span></a><span class="sd"> Get a value for a nested dictionary.</span>
|
||||
</span><span id="nested_get-423"><a href="#nested_get-423"><span class="linenos">423</span></a>
|
||||
</span><span id="nested_get-424"><a href="#nested_get-424"><span class="linenos">424</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="nested_get-425"><a href="#nested_get-425"><span class="linenos">425</span></a><span class="sd"> d: the dictionary to search.</span>
|
||||
</span><span id="nested_get-426"><a href="#nested_get-426"><span class="linenos">426</span></a><span class="sd"> *path: tuples of (name, key), where:</span>
|
||||
</span><span id="nested_get-427"><a href="#nested_get-427"><span class="linenos">427</span></a><span class="sd"> `key` is the key in the dictionary to get.</span>
|
||||
</span><span id="nested_get-428"><a href="#nested_get-428"><span class="linenos">428</span></a><span class="sd"> `name` is a string to use in the error if `key` isn't found.</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="nested_get-424"><a href="#nested_get-424"><span class="linenos">424</span></a><span class="k">def</span> <span class="nf">nested_get</span><span class="p">(</span>
|
||||
</span><span id="nested_get-425"><a href="#nested_get-425"><span class="linenos">425</span></a> <span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="o">*</span><span class="n">path</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">str</span><span class="p">],</span> <span class="n">raise_on_missing</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">True</span>
|
||||
</span><span id="nested_get-426"><a href="#nested_get-426"><span class="linenos">426</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">]:</span>
|
||||
</span><span id="nested_get-427"><a href="#nested_get-427"><span class="linenos">427</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="nested_get-428"><a href="#nested_get-428"><span class="linenos">428</span></a><span class="sd"> Get a value for a nested dictionary.</span>
|
||||
</span><span id="nested_get-429"><a href="#nested_get-429"><span class="linenos">429</span></a>
|
||||
</span><span id="nested_get-430"><a href="#nested_get-430"><span class="linenos">430</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="nested_get-431"><a href="#nested_get-431"><span class="linenos">431</span></a><span class="sd"> The value or None if it doesn't exist.</span>
|
||||
</span><span id="nested_get-432"><a href="#nested_get-432"><span class="linenos">432</span></a><span class="sd"> """</span>
|
||||
</span><span id="nested_get-433"><a href="#nested_get-433"><span class="linenos">433</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">path</span><span class="p">:</span>
|
||||
</span><span id="nested_get-434"><a href="#nested_get-434"><span class="linenos">434</span></a> <span class="n">d</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="nested_get-435"><a href="#nested_get-435"><span class="linenos">435</span></a> <span class="k">if</span> <span class="n">d</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="nested_get-436"><a href="#nested_get-436"><span class="linenos">436</span></a> <span class="k">if</span> <span class="n">raise_on_missing</span><span class="p">:</span>
|
||||
</span><span id="nested_get-437"><a href="#nested_get-437"><span class="linenos">437</span></a> <span class="n">name</span> <span class="o">=</span> <span class="s2">"table"</span> <span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="s2">"this"</span> <span class="k">else</span> <span class="n">name</span>
|
||||
</span><span id="nested_get-438"><a href="#nested_get-438"><span class="linenos">438</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">: </span><span class="si">{</span><span class="n">key</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="nested_get-439"><a href="#nested_get-439"><span class="linenos">439</span></a> <span class="k">return</span> <span class="kc">None</span>
|
||||
</span><span id="nested_get-440"><a href="#nested_get-440"><span class="linenos">440</span></a>
|
||||
</span><span id="nested_get-441"><a href="#nested_get-441"><span class="linenos">441</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="nested_get-430"><a href="#nested_get-430"><span class="linenos">430</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="nested_get-431"><a href="#nested_get-431"><span class="linenos">431</span></a><span class="sd"> d: the dictionary to search.</span>
|
||||
</span><span id="nested_get-432"><a href="#nested_get-432"><span class="linenos">432</span></a><span class="sd"> *path: tuples of (name, key), where:</span>
|
||||
</span><span id="nested_get-433"><a href="#nested_get-433"><span class="linenos">433</span></a><span class="sd"> `key` is the key in the dictionary to get.</span>
|
||||
</span><span id="nested_get-434"><a href="#nested_get-434"><span class="linenos">434</span></a><span class="sd"> `name` is a string to use in the error if `key` isn't found.</span>
|
||||
</span><span id="nested_get-435"><a href="#nested_get-435"><span class="linenos">435</span></a>
|
||||
</span><span id="nested_get-436"><a href="#nested_get-436"><span class="linenos">436</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="nested_get-437"><a href="#nested_get-437"><span class="linenos">437</span></a><span class="sd"> The value or None if it doesn't exist.</span>
|
||||
</span><span id="nested_get-438"><a href="#nested_get-438"><span class="linenos">438</span></a><span class="sd"> """</span>
|
||||
</span><span id="nested_get-439"><a href="#nested_get-439"><span class="linenos">439</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">path</span><span class="p">:</span>
|
||||
</span><span id="nested_get-440"><a href="#nested_get-440"><span class="linenos">440</span></a> <span class="n">d</span> <span class="o">=</span> <span class="n">d</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> <span class="c1"># type: ignore</span>
|
||||
</span><span id="nested_get-441"><a href="#nested_get-441"><span class="linenos">441</span></a> <span class="k">if</span> <span class="n">d</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||||
</span><span id="nested_get-442"><a href="#nested_get-442"><span class="linenos">442</span></a> <span class="k">if</span> <span class="n">raise_on_missing</span><span class="p">:</span>
|
||||
</span><span id="nested_get-443"><a href="#nested_get-443"><span class="linenos">443</span></a> <span class="n">name</span> <span class="o">=</span> <span class="s2">"table"</span> <span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="s2">"this"</span> <span class="k">else</span> <span class="n">name</span>
|
||||
</span><span id="nested_get-444"><a href="#nested_get-444"><span class="linenos">444</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Unknown </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">: </span><span class="si">{</span><span class="n">key</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
|
||||
</span><span id="nested_get-445"><a href="#nested_get-445"><span class="linenos">445</span></a> <span class="k">return</span> <span class="kc">None</span>
|
||||
</span><span id="nested_get-446"><a href="#nested_get-446"><span class="linenos">446</span></a>
|
||||
</span><span id="nested_get-447"><a href="#nested_get-447"><span class="linenos">447</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
@ -1749,41 +1761,41 @@ are assumed to be visible. The nesting should mirror that of the schema:
|
|||
|
||||
</div>
|
||||
<a class="headerlink" href="#nested_set"></a>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="nested_set-444"><a href="#nested_set-444"><span class="linenos">444</span></a><span class="k">def</span> <span class="nf">nested_set</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Sequence</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="nested_set-445"><a href="#nested_set-445"><span class="linenos">445</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="nested_set-446"><a href="#nested_set-446"><span class="linenos">446</span></a><span class="sd"> In-place set a value for a nested dictionary</span>
|
||||
</span><span id="nested_set-447"><a href="#nested_set-447"><span class="linenos">447</span></a>
|
||||
</span><span id="nested_set-448"><a href="#nested_set-448"><span class="linenos">448</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="nested_set-449"><a href="#nested_set-449"><span class="linenos">449</span></a><span class="sd"> >>> nested_set({}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="nested_set-450"><a href="#nested_set-450"><span class="linenos">450</span></a><span class="sd"> {'top_key': {'second_key': 'value'}}</span>
|
||||
</span><span id="nested_set-451"><a href="#nested_set-451"><span class="linenos">451</span></a>
|
||||
</span><span id="nested_set-452"><a href="#nested_set-452"><span class="linenos">452</span></a><span class="sd"> >>> nested_set({"top_key": {"third_key": "third_value"}}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="nested_set-453"><a href="#nested_set-453"><span class="linenos">453</span></a><span class="sd"> {'top_key': {'third_key': 'third_value', 'second_key': 'value'}}</span>
|
||||
</span><span id="nested_set-454"><a href="#nested_set-454"><span class="linenos">454</span></a>
|
||||
</span><span id="nested_set-455"><a href="#nested_set-455"><span class="linenos">455</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="nested_set-456"><a href="#nested_set-456"><span class="linenos">456</span></a><span class="sd"> d: dictionary to update.</span>
|
||||
</span><span id="nested_set-457"><a href="#nested_set-457"><span class="linenos">457</span></a><span class="sd"> keys: the keys that makeup the path to `value`.</span>
|
||||
</span><span id="nested_set-458"><a href="#nested_set-458"><span class="linenos">458</span></a><span class="sd"> value: the value to set in the dictionary for the given key path.</span>
|
||||
</span><span id="nested_set-459"><a href="#nested_set-459"><span class="linenos">459</span></a>
|
||||
</span><span id="nested_set-460"><a href="#nested_set-460"><span class="linenos">460</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="nested_set-461"><a href="#nested_set-461"><span class="linenos">461</span></a><span class="sd"> The (possibly) updated dictionary.</span>
|
||||
</span><span id="nested_set-462"><a href="#nested_set-462"><span class="linenos">462</span></a><span class="sd"> """</span>
|
||||
</span><span id="nested_set-463"><a href="#nested_set-463"><span class="linenos">463</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">keys</span><span class="p">:</span>
|
||||
</span><span id="nested_set-464"><a href="#nested_set-464"><span class="linenos">464</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
<div class="pdoc-code codehilite"><pre><span></span><span id="nested_set-450"><a href="#nested_set-450"><span class="linenos">450</span></a><span class="k">def</span> <span class="nf">nested_set</span><span class="p">(</span><span class="n">d</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">,</span> <span class="n">keys</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Sequence</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">value</span><span class="p">:</span> <span class="n">t</span><span class="o">.</span><span class="n">Any</span><span class="p">)</span> <span class="o">-></span> <span class="n">t</span><span class="o">.</span><span class="n">Dict</span><span class="p">:</span>
|
||||
</span><span id="nested_set-451"><a href="#nested_set-451"><span class="linenos">451</span></a><span class="w"> </span><span class="sd">"""</span>
|
||||
</span><span id="nested_set-452"><a href="#nested_set-452"><span class="linenos">452</span></a><span class="sd"> In-place set a value for a nested dictionary</span>
|
||||
</span><span id="nested_set-453"><a href="#nested_set-453"><span class="linenos">453</span></a>
|
||||
</span><span id="nested_set-454"><a href="#nested_set-454"><span class="linenos">454</span></a><span class="sd"> Example:</span>
|
||||
</span><span id="nested_set-455"><a href="#nested_set-455"><span class="linenos">455</span></a><span class="sd"> >>> nested_set({}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="nested_set-456"><a href="#nested_set-456"><span class="linenos">456</span></a><span class="sd"> {'top_key': {'second_key': 'value'}}</span>
|
||||
</span><span id="nested_set-457"><a href="#nested_set-457"><span class="linenos">457</span></a>
|
||||
</span><span id="nested_set-458"><a href="#nested_set-458"><span class="linenos">458</span></a><span class="sd"> >>> nested_set({"top_key": {"third_key": "third_value"}}, ["top_key", "second_key"], "value")</span>
|
||||
</span><span id="nested_set-459"><a href="#nested_set-459"><span class="linenos">459</span></a><span class="sd"> {'top_key': {'third_key': 'third_value', 'second_key': 'value'}}</span>
|
||||
</span><span id="nested_set-460"><a href="#nested_set-460"><span class="linenos">460</span></a>
|
||||
</span><span id="nested_set-461"><a href="#nested_set-461"><span class="linenos">461</span></a><span class="sd"> Args:</span>
|
||||
</span><span id="nested_set-462"><a href="#nested_set-462"><span class="linenos">462</span></a><span class="sd"> d: dictionary to update.</span>
|
||||
</span><span id="nested_set-463"><a href="#nested_set-463"><span class="linenos">463</span></a><span class="sd"> keys: the keys that makeup the path to `value`.</span>
|
||||
</span><span id="nested_set-464"><a href="#nested_set-464"><span class="linenos">464</span></a><span class="sd"> value: the value to set in the dictionary for the given key path.</span>
|
||||
</span><span id="nested_set-465"><a href="#nested_set-465"><span class="linenos">465</span></a>
|
||||
</span><span id="nested_set-466"><a href="#nested_set-466"><span class="linenos">466</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">keys</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="nested_set-467"><a href="#nested_set-467"><span class="linenos">467</span></a> <span class="n">d</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="mi">0</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="nested_set-468"><a href="#nested_set-468"><span class="linenos">468</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="nested_set-469"><a href="#nested_set-469"><span class="linenos">469</span></a>
|
||||
</span><span id="nested_set-470"><a href="#nested_set-470"><span class="linenos">470</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">d</span>
|
||||
</span><span id="nested_set-471"><a href="#nested_set-471"><span class="linenos">471</span></a> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">keys</span><span class="p">[:</span><span class="o">-</span><span class="mi">1</span><span class="p">]:</span>
|
||||
</span><span id="nested_set-472"><a href="#nested_set-472"><span class="linenos">472</span></a> <span class="k">if</span> <span class="n">key</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">subd</span><span class="p">:</span>
|
||||
</span><span id="nested_set-473"><a href="#nested_set-473"><span class="linenos">473</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="o">.</span><span class="n">setdefault</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="p">{})</span>
|
||||
</span><span id="nested_set-474"><a href="#nested_set-474"><span class="linenos">474</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="nested_set-475"><a href="#nested_set-475"><span class="linenos">475</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>
|
||||
</span><span id="nested_set-476"><a href="#nested_set-476"><span class="linenos">476</span></a>
|
||||
</span><span id="nested_set-477"><a href="#nested_set-477"><span class="linenos">477</span></a> <span class="n">subd</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="nested_set-478"><a href="#nested_set-478"><span class="linenos">478</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="nested_set-466"><a href="#nested_set-466"><span class="linenos">466</span></a><span class="sd"> Returns:</span>
|
||||
</span><span id="nested_set-467"><a href="#nested_set-467"><span class="linenos">467</span></a><span class="sd"> The (possibly) updated dictionary.</span>
|
||||
</span><span id="nested_set-468"><a href="#nested_set-468"><span class="linenos">468</span></a><span class="sd"> """</span>
|
||||
</span><span id="nested_set-469"><a href="#nested_set-469"><span class="linenos">469</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">keys</span><span class="p">:</span>
|
||||
</span><span id="nested_set-470"><a href="#nested_set-470"><span class="linenos">470</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="nested_set-471"><a href="#nested_set-471"><span class="linenos">471</span></a>
|
||||
</span><span id="nested_set-472"><a href="#nested_set-472"><span class="linenos">472</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">keys</span><span class="p">)</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
|
||||
</span><span id="nested_set-473"><a href="#nested_set-473"><span class="linenos">473</span></a> <span class="n">d</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="mi">0</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="nested_set-474"><a href="#nested_set-474"><span class="linenos">474</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span><span id="nested_set-475"><a href="#nested_set-475"><span class="linenos">475</span></a>
|
||||
</span><span id="nested_set-476"><a href="#nested_set-476"><span class="linenos">476</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">d</span>
|
||||
</span><span id="nested_set-477"><a href="#nested_set-477"><span class="linenos">477</span></a> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">keys</span><span class="p">[:</span><span class="o">-</span><span class="mi">1</span><span class="p">]:</span>
|
||||
</span><span id="nested_set-478"><a href="#nested_set-478"><span class="linenos">478</span></a> <span class="k">if</span> <span class="n">key</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">subd</span><span class="p">:</span>
|
||||
</span><span id="nested_set-479"><a href="#nested_set-479"><span class="linenos">479</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="o">.</span><span class="n">setdefault</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="p">{})</span>
|
||||
</span><span id="nested_set-480"><a href="#nested_set-480"><span class="linenos">480</span></a> <span class="k">else</span><span class="p">:</span>
|
||||
</span><span id="nested_set-481"><a href="#nested_set-481"><span class="linenos">481</span></a> <span class="n">subd</span> <span class="o">=</span> <span class="n">subd</span><span class="p">[</span><span class="n">key</span><span class="p">]</span>
|
||||
</span><span id="nested_set-482"><a href="#nested_set-482"><span class="linenos">482</span></a>
|
||||
</span><span id="nested_set-483"><a href="#nested_set-483"><span class="linenos">483</span></a> <span class="n">subd</span><span class="p">[</span><span class="n">keys</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]]</span> <span class="o">=</span> <span class="n">value</span>
|
||||
</span><span id="nested_set-484"><a href="#nested_set-484"><span class="linenos">484</span></a> <span class="k">return</span> <span class="n">d</span>
|
||||
</span></pre></div>
|
||||
|
||||
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -84,58 +84,17 @@ def parse(sql: str, read: DialectType = None, **opts) -> t.List[t.Optional[Expre
|
|||
|
||||
|
||||
@t.overload
|
||||
def parse_one(
|
||||
sql: str,
|
||||
read: None = None,
|
||||
into: t.Type[E] = ...,
|
||||
**opts,
|
||||
) -> E:
|
||||
def parse_one(sql: str, *, into: t.Type[E], **opts) -> E:
|
||||
...
|
||||
|
||||
|
||||
@t.overload
|
||||
def parse_one(
|
||||
sql: str,
|
||||
read: DialectType,
|
||||
into: t.Type[E],
|
||||
**opts,
|
||||
) -> E:
|
||||
...
|
||||
|
||||
|
||||
@t.overload
|
||||
def parse_one(
|
||||
sql: str,
|
||||
read: None = None,
|
||||
into: t.Union[str, t.Collection[t.Union[str, t.Type[Expression]]]] = ...,
|
||||
**opts,
|
||||
) -> Expression:
|
||||
...
|
||||
|
||||
|
||||
@t.overload
|
||||
def parse_one(
|
||||
sql: str,
|
||||
read: DialectType,
|
||||
into: t.Union[str, t.Collection[t.Union[str, t.Type[Expression]]]],
|
||||
**opts,
|
||||
) -> Expression:
|
||||
...
|
||||
|
||||
|
||||
@t.overload
|
||||
def parse_one(
|
||||
sql: str,
|
||||
**opts,
|
||||
) -> Expression:
|
||||
def parse_one(sql: str, **opts) -> Expression:
|
||||
...
|
||||
|
||||
|
||||
def parse_one(
|
||||
sql: str,
|
||||
read: DialectType = None,
|
||||
into: t.Optional[exp.IntoType] = None,
|
||||
**opts,
|
||||
sql: str, read: DialectType = None, into: t.Optional[exp.IntoType] = None, **opts
|
||||
) -> Expression:
|
||||
"""
|
||||
Parses the given SQL string and returns a syntax tree for the first parsed SQL statement.
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ Currently many of the common operations are covered and more functionality will
|
|||
## Instructions
|
||||
* [Install SQLGlot](https://github.com/tobymao/sqlglot/blob/main/README.md#install) and that is all that is required to just generate SQL. [The examples](#examples) show generating SQL and then executing that SQL on a specific engine and that will require that engine's client library.
|
||||
* Find/replace all `from pyspark.sql` with `from sqlglot.dataframe`.
|
||||
* Prior to any `spark.read.table` or `spark.table` run `sqlglot.schema.add_table('<table_name>', <column_structure>)`.
|
||||
* Prior to any `spark.read.table` or `spark.table` run `sqlglot.schema.add_table('<table_name>', <column_structure>, dialect="spark")`.
|
||||
* The column structure can be defined the following ways:
|
||||
* Dictionary where the keys are column names and values are string of the Spark SQL type name.
|
||||
* Ex: `{'cola': 'string', 'colb': 'int'}`
|
||||
|
|
@ -33,12 +33,16 @@ import sqlglot
|
|||
from sqlglot.dataframe.sql.session import SparkSession
|
||||
from sqlglot.dataframe.sql import functions as F
|
||||
|
||||
sqlglot.schema.add_table('employee', {
|
||||
sqlglot.schema.add_table(
|
||||
'employee',
|
||||
{
|
||||
'employee_id': 'INT',
|
||||
'fname': 'STRING',
|
||||
'lname': 'STRING',
|
||||
'age': 'INT',
|
||||
}) # Register the table structure prior to reading from the table
|
||||
},
|
||||
dialect="spark",
|
||||
) # Register the table structure prior to reading from the table
|
||||
|
||||
spark = SparkSession()
|
||||
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@ import typing as t
|
|||
import sqlglot
|
||||
from sqlglot import expressions as exp
|
||||
from sqlglot.dataframe.sql.types import DataType
|
||||
from sqlglot.dialects import Spark
|
||||
from sqlglot.helper import flatten, is_iterable
|
||||
|
||||
if t.TYPE_CHECKING:
|
||||
|
|
@ -22,6 +23,10 @@ class Column:
|
|||
expression = sqlglot.maybe_parse(expression, dialect="spark")
|
||||
if expression is None:
|
||||
raise ValueError(f"Could not parse {expression}")
|
||||
|
||||
if isinstance(expression, exp.Column):
|
||||
expression.transform(Spark.normalize_identifier, copy=False)
|
||||
|
||||
self.expression: exp.Expression = expression
|
||||
|
||||
def __repr__(self):
|
||||
|
|
|
|||
|
|
@ -316,6 +316,7 @@ class DataFrame:
|
|||
expression.alias_or_name: expression.type.sql("spark")
|
||||
for expression in select_expression.expressions
|
||||
},
|
||||
dialect="spark",
|
||||
)
|
||||
cache_storage_level = select_expression.args["cache_storage_level"]
|
||||
options = [
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@ import typing as t
|
|||
from sqlglot import expressions as exp
|
||||
from sqlglot.dataframe.sql.column import Column
|
||||
from sqlglot.dataframe.sql.util import get_tables_from_expression_with_join
|
||||
from sqlglot.dialects import Spark
|
||||
from sqlglot.helper import ensure_list
|
||||
|
||||
NORMALIZE_INPUT = t.TypeVar("NORMALIZE_INPUT", bound=t.Union[str, exp.Expression, Column])
|
||||
|
|
@ -19,6 +20,7 @@ def normalize(spark: SparkSession, expression_context: exp.Select, expr: t.List[
|
|||
for expression in expressions:
|
||||
identifiers = expression.find_all(exp.Identifier)
|
||||
for identifier in identifiers:
|
||||
Spark.normalize_identifier(identifier)
|
||||
replace_alias_name_with_cte_name(spark, expression_context, identifier)
|
||||
replace_branch_and_sequence_ids_with_cte_name(spark, expression_context, identifier)
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,8 @@ import typing as t
|
|||
|
||||
import sqlglot
|
||||
from sqlglot import expressions as exp
|
||||
from sqlglot.helper import object_to_dict, should_identify
|
||||
from sqlglot.dialects import Spark
|
||||
from sqlglot.helper import object_to_dict
|
||||
|
||||
if t.TYPE_CHECKING:
|
||||
from sqlglot.dataframe.sql.dataframe import DataFrame
|
||||
|
|
@ -18,17 +19,14 @@ class DataFrameReader:
|
|||
def table(self, tableName: str) -> DataFrame:
|
||||
from sqlglot.dataframe.sql.dataframe import DataFrame
|
||||
|
||||
sqlglot.schema.add_table(tableName)
|
||||
sqlglot.schema.add_table(tableName, dialect="spark")
|
||||
|
||||
return DataFrame(
|
||||
self.spark,
|
||||
exp.Select()
|
||||
.from_(tableName)
|
||||
.from_(exp.to_table(tableName, dialect="spark").transform(Spark.normalize_identifier))
|
||||
.select(
|
||||
*(
|
||||
column if should_identify(column, "safe") else f'"{column}"'
|
||||
for column in sqlglot.schema.column_names(tableName)
|
||||
)
|
||||
*(column for column in sqlglot.schema.column_names(tableName, dialect="spark"))
|
||||
),
|
||||
)
|
||||
|
||||
|
|
@ -73,7 +71,7 @@ class DataFrameWriter:
|
|||
)
|
||||
df = self._df.copy(output_expression_container=output_expression_container)
|
||||
if self._by_name:
|
||||
columns = sqlglot.schema.column_names(tableName, only_visible=True)
|
||||
columns = sqlglot.schema.column_names(tableName, only_visible=True, dialect="spark")
|
||||
df = df._convert_leaf_to_cte().select(*columns)
|
||||
|
||||
return self.copy(_df=df)
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ import re
|
|||
import typing as t
|
||||
|
||||
from sqlglot import exp, generator, parser, tokens, transforms
|
||||
from sqlglot._typing import E
|
||||
from sqlglot.dialects.dialect import (
|
||||
Dialect,
|
||||
datestrtodate_sql,
|
||||
|
|
@ -106,6 +107,9 @@ def _unqualify_unnest(expression: exp.Expression) -> exp.Expression:
|
|||
class BigQuery(Dialect):
|
||||
UNNEST_COLUMN_ONLY = True
|
||||
|
||||
# https://cloud.google.com/bigquery/docs/reference/standard-sql/lexical#case_sensitivity
|
||||
RESOLVES_IDENTIFIERS_AS_UPPERCASE = None
|
||||
|
||||
TIME_MAPPING = {
|
||||
"%D": "%m/%d/%y",
|
||||
}
|
||||
|
|
@ -126,6 +130,20 @@ class BigQuery(Dialect):
|
|||
"TZH": "%z",
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def normalize_identifier(cls, expression: E) -> E:
|
||||
# In BigQuery, CTEs aren't case-sensitive, but table names are (by default, at least).
|
||||
# The following check is essentially a heuristic to detect tables based on whether or
|
||||
# not they're qualified.
|
||||
if (
|
||||
isinstance(expression, exp.Identifier)
|
||||
and not (isinstance(expression.parent, exp.Table) and expression.parent.db)
|
||||
and not expression.meta.get("is_table")
|
||||
):
|
||||
expression.set("this", expression.this.lower())
|
||||
|
||||
return expression
|
||||
|
||||
class Tokenizer(tokens.Tokenizer):
|
||||
QUOTES = ["'", '"', '"""', "'''"]
|
||||
COMMENTS = ["--", "#", ("/*", "*/")]
|
||||
|
|
@ -176,6 +194,7 @@ class BigQuery(Dialect):
|
|||
"DATETIME_ADD": parse_date_delta_with_interval(exp.DatetimeAdd),
|
||||
"DATETIME_SUB": parse_date_delta_with_interval(exp.DatetimeSub),
|
||||
"DIV": lambda args: exp.IntDiv(this=seq_get(args, 0), expression=seq_get(args, 1)),
|
||||
"GENERATE_ARRAY": exp.GenerateSeries.from_arg_list,
|
||||
"PARSE_DATE": lambda args: format_time_lambda(exp.StrToDate, "bigquery")(
|
||||
[seq_get(args, 1), seq_get(args, 0)]
|
||||
),
|
||||
|
|
@ -201,6 +220,7 @@ class BigQuery(Dialect):
|
|||
"TIME_SUB": parse_date_delta_with_interval(exp.TimeSub),
|
||||
"TIMESTAMP_ADD": parse_date_delta_with_interval(exp.TimestampAdd),
|
||||
"TIMESTAMP_SUB": parse_date_delta_with_interval(exp.TimestampSub),
|
||||
"TO_JSON_STRING": exp.JSONFormat.from_arg_list,
|
||||
}
|
||||
|
||||
FUNCTION_PARSERS = {
|
||||
|
|
@ -289,6 +309,8 @@ class BigQuery(Dialect):
|
|||
exp.DateDiff: lambda self, e: f"DATE_DIFF({self.sql(e, 'this')}, {self.sql(e, 'expression')}, {self.sql(e.args.get('unit', 'DAY'))})",
|
||||
exp.DateStrToDate: datestrtodate_sql,
|
||||
exp.DateTrunc: lambda self, e: self.func("DATE_TRUNC", e.this, e.text("unit")),
|
||||
exp.JSONFormat: rename_func("TO_JSON_STRING"),
|
||||
exp.GenerateSeries: rename_func("GENERATE_ARRAY"),
|
||||
exp.GroupConcat: rename_func("STRING_AGG"),
|
||||
exp.ILike: no_ilike_sql,
|
||||
exp.IntDiv: rename_func("DIV"),
|
||||
|
|
|
|||
|
|
@ -345,7 +345,7 @@ class ClickHouse(Dialect):
|
|||
"CONCAT",
|
||||
*[
|
||||
exp.func("if", e.is_(exp.null()), e, exp.cast(e, "text"))
|
||||
for e in expression.expressions
|
||||
for e in t.cast(t.List[exp.Condition], expression.expressions)
|
||||
],
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ import typing as t
|
|||
from enum import Enum
|
||||
|
||||
from sqlglot import exp
|
||||
from sqlglot._typing import E
|
||||
from sqlglot.generator import Generator
|
||||
from sqlglot.helper import flatten, seq_get
|
||||
from sqlglot.parser import Parser
|
||||
|
|
@ -11,14 +12,6 @@ from sqlglot.time import format_time
|
|||
from sqlglot.tokens import Token, Tokenizer, TokenType
|
||||
from sqlglot.trie import new_trie
|
||||
|
||||
if t.TYPE_CHECKING:
|
||||
from sqlglot._typing import E
|
||||
|
||||
|
||||
# Only Snowflake is currently known to resolve unquoted identifiers as uppercase.
|
||||
# https://docs.snowflake.com/en/sql-reference/identifiers-syntax
|
||||
RESOLVES_IDENTIFIERS_AS_UPPERCASE = {"snowflake"}
|
||||
|
||||
|
||||
class Dialects(str, Enum):
|
||||
DIALECT = ""
|
||||
|
|
@ -117,6 +110,9 @@ class _Dialect(type):
|
|||
"IDENTIFIER_ESCAPE": klass.tokenizer_class.IDENTIFIER_ESCAPES[0],
|
||||
}
|
||||
|
||||
if enum not in ("", "bigquery"):
|
||||
dialect_properties["SELECT_KINDS"] = ()
|
||||
|
||||
# Pass required dialect properties to the tokenizer, parser and generator classes
|
||||
for subclass in (klass.tokenizer_class, klass.parser_class, klass.generator_class):
|
||||
for name, value in dialect_properties.items():
|
||||
|
|
@ -126,6 +122,8 @@ class _Dialect(type):
|
|||
if not klass.STRICT_STRING_CONCAT:
|
||||
klass.parser_class.BITWISE[TokenType.DPIPE] = exp.SafeDPipe
|
||||
|
||||
klass.generator_class.can_identify = klass.can_identify
|
||||
|
||||
return klass
|
||||
|
||||
|
||||
|
|
@ -139,6 +137,10 @@ class Dialect(metaclass=_Dialect):
|
|||
# Determines whether or not the table alias comes after tablesample
|
||||
ALIAS_POST_TABLESAMPLE = False
|
||||
|
||||
# Determines whether or not unquoted identifiers are resolved as uppercase
|
||||
# When set to None, it means that the dialect treats all identifiers as case-insensitive
|
||||
RESOLVES_IDENTIFIERS_AS_UPPERCASE: t.Optional[bool] = False
|
||||
|
||||
# Determines whether or not an unquoted identifier can start with a digit
|
||||
IDENTIFIERS_CAN_START_WITH_DIGIT = False
|
||||
|
||||
|
|
@ -213,6 +215,66 @@ class Dialect(metaclass=_Dialect):
|
|||
|
||||
return expression
|
||||
|
||||
@classmethod
|
||||
def normalize_identifier(cls, expression: E) -> E:
|
||||
"""
|
||||
Normalizes an unquoted identifier to either lower or upper case, thus essentially
|
||||
making it case-insensitive. If a dialect treats all identifiers as case-insensitive,
|
||||
they will be normalized regardless of being quoted or not.
|
||||
"""
|
||||
if isinstance(expression, exp.Identifier) and (
|
||||
not expression.quoted or cls.RESOLVES_IDENTIFIERS_AS_UPPERCASE is None
|
||||
):
|
||||
expression.set(
|
||||
"this",
|
||||
expression.this.upper()
|
||||
if cls.RESOLVES_IDENTIFIERS_AS_UPPERCASE
|
||||
else expression.this.lower(),
|
||||
)
|
||||
|
||||
return expression
|
||||
|
||||
@classmethod
|
||||
def case_sensitive(cls, text: str) -> bool:
|
||||
"""Checks if text contains any case sensitive characters, based on the dialect's rules."""
|
||||
if cls.RESOLVES_IDENTIFIERS_AS_UPPERCASE is None:
|
||||
return False
|
||||
|
||||
unsafe = str.islower if cls.RESOLVES_IDENTIFIERS_AS_UPPERCASE else str.isupper
|
||||
return any(unsafe(char) for char in text)
|
||||
|
||||
@classmethod
|
||||
def can_identify(cls, text: str, identify: str | bool = "safe") -> bool:
|
||||
"""Checks if text can be identified given an identify option.
|
||||
|
||||
Args:
|
||||
text: The text to check.
|
||||
identify:
|
||||
"always" or `True`: Always returns true.
|
||||
"safe": True if the identifier is case-insensitive.
|
||||
|
||||
Returns:
|
||||
Whether or not the given text can be identified.
|
||||
"""
|
||||
if identify is True or identify == "always":
|
||||
return True
|
||||
|
||||
if identify == "safe":
|
||||
return not cls.case_sensitive(text)
|
||||
|
||||
return False
|
||||
|
||||
@classmethod
|
||||
def quote_identifier(cls, expression: E, identify: bool = True) -> E:
|
||||
if isinstance(expression, exp.Identifier):
|
||||
name = expression.this
|
||||
expression.set(
|
||||
"quoted",
|
||||
identify or cls.case_sensitive(name) or not exp.SAFE_IDENTIFIER_RE.match(name),
|
||||
)
|
||||
|
||||
return expression
|
||||
|
||||
def parse(self, sql: str, **opts) -> t.List[t.Optional[exp.Expression]]:
|
||||
return self.parser(**opts).parse(self.tokenize(sql), sql)
|
||||
|
||||
|
|
|
|||
|
|
@ -85,9 +85,17 @@ def _regexp_extract_sql(self: generator.Generator, expression: exp.RegexpExtract
|
|||
)
|
||||
|
||||
|
||||
def _json_format_sql(self: generator.Generator, expression: exp.JSONFormat) -> str:
|
||||
sql = self.func("TO_JSON", expression.this, expression.args.get("options"))
|
||||
return f"CAST({sql} AS TEXT)"
|
||||
|
||||
|
||||
class DuckDB(Dialect):
|
||||
NULL_ORDERING = "nulls_are_last"
|
||||
|
||||
# https://duckdb.org/docs/sql/introduction.html#creating-a-new-table
|
||||
RESOLVES_IDENTIFIERS_AS_UPPERCASE = None
|
||||
|
||||
class Tokenizer(tokens.Tokenizer):
|
||||
KEYWORDS = {
|
||||
**tokens.Tokenizer.KEYWORDS,
|
||||
|
|
@ -167,7 +175,7 @@ class DuckDB(Dialect):
|
|||
**generator.Generator.TRANSFORMS,
|
||||
exp.ApproxDistinct: approx_count_distinct_sql,
|
||||
exp.Array: lambda self, e: self.func("ARRAY", e.expressions[0])
|
||||
if isinstance(seq_get(e.expressions, 0), exp.Select)
|
||||
if e.expressions and e.expressions[0].find(exp.Select)
|
||||
else rename_func("LIST_VALUE")(self, e),
|
||||
exp.ArraySize: rename_func("ARRAY_LENGTH"),
|
||||
exp.ArraySort: _array_sort_sql,
|
||||
|
|
@ -192,6 +200,7 @@ class DuckDB(Dialect):
|
|||
exp.IntDiv: lambda self, e: self.binary(e, "//"),
|
||||
exp.JSONExtract: arrow_json_extract_sql,
|
||||
exp.JSONExtractScalar: arrow_json_extract_scalar_sql,
|
||||
exp.JSONFormat: _json_format_sql,
|
||||
exp.JSONBExtract: arrow_json_extract_sql,
|
||||
exp.JSONBExtractScalar: arrow_json_extract_scalar_sql,
|
||||
exp.LogicalOr: rename_func("BOOL_OR"),
|
||||
|
|
|
|||
|
|
@ -86,13 +86,17 @@ def _date_diff_sql(self: generator.Generator, expression: exp.DateDiff) -> str:
|
|||
|
||||
def _json_format_sql(self: generator.Generator, expression: exp.JSONFormat) -> str:
|
||||
this = expression.this
|
||||
if not this.type:
|
||||
from sqlglot.optimizer.annotate_types import annotate_types
|
||||
if isinstance(this, exp.Cast) and this.is_type("json") and this.this.is_string:
|
||||
# Since FROM_JSON requires a nested type, we always wrap the json string with
|
||||
# an array to ensure that "naked" strings like "'a'" will be handled correctly
|
||||
wrapped_json = exp.Literal.string(f"[{this.this.name}]")
|
||||
|
||||
annotate_types(this)
|
||||
from_json = self.func("FROM_JSON", wrapped_json, self.func("SCHEMA_OF_JSON", wrapped_json))
|
||||
to_json = self.func("TO_JSON", from_json)
|
||||
|
||||
# This strips the [, ] delimiters of the dummy array printed by TO_JSON
|
||||
return self.func("REGEXP_EXTRACT", to_json, "'^.(.*).$'", "1")
|
||||
|
||||
if this.type.is_type("json"):
|
||||
return self.sql(this)
|
||||
return self.func("TO_JSON", this, expression.args.get("options"))
|
||||
|
||||
|
||||
|
|
@ -153,6 +157,9 @@ class Hive(Dialect):
|
|||
ALIAS_POST_TABLESAMPLE = True
|
||||
IDENTIFIERS_CAN_START_WITH_DIGIT = True
|
||||
|
||||
# https://spark.apache.org/docs/latest/sql-ref-identifier.html#description
|
||||
RESOLVES_IDENTIFIERS_AS_UPPERCASE = None
|
||||
|
||||
TIME_MAPPING = {
|
||||
"y": "%Y",
|
||||
"Y": "%Y",
|
||||
|
|
@ -268,9 +275,9 @@ class Hive(Dialect):
|
|||
|
||||
QUERY_MODIFIER_PARSERS = {
|
||||
**parser.Parser.QUERY_MODIFIER_PARSERS,
|
||||
"distribute": lambda self: self._parse_sort(exp.Distribute, "DISTRIBUTE", "BY"),
|
||||
"sort": lambda self: self._parse_sort(exp.Sort, "SORT", "BY"),
|
||||
"cluster": lambda self: self._parse_sort(exp.Cluster, "CLUSTER", "BY"),
|
||||
"cluster": lambda self: self._parse_sort(exp.Cluster, TokenType.CLUSTER_BY),
|
||||
"distribute": lambda self: self._parse_sort(exp.Distribute, TokenType.DISTRIBUTE_BY),
|
||||
"sort": lambda self: self._parse_sort(exp.Sort, TokenType.SORT_BY),
|
||||
}
|
||||
|
||||
def _parse_types(
|
||||
|
|
|
|||
|
|
@ -123,6 +123,8 @@ class MySQL(Dialect):
|
|||
KEYWORDS = {
|
||||
**tokens.Tokenizer.KEYWORDS,
|
||||
"CHARSET": TokenType.CHARACTER_SET,
|
||||
"FORCE": TokenType.FORCE,
|
||||
"IGNORE": TokenType.IGNORE,
|
||||
"LONGBLOB": TokenType.LONGBLOB,
|
||||
"LONGTEXT": TokenType.LONGTEXT,
|
||||
"MEDIUMBLOB": TokenType.MEDIUMBLOB,
|
||||
|
|
@ -180,6 +182,9 @@ class MySQL(Dialect):
|
|||
|
||||
class Parser(parser.Parser):
|
||||
FUNC_TOKENS = {*parser.Parser.FUNC_TOKENS, TokenType.SCHEMA, TokenType.DATABASE}
|
||||
TABLE_ALIAS_TOKENS = (
|
||||
parser.Parser.TABLE_ALIAS_TOKENS - parser.Parser.TABLE_INDEX_HINT_TOKENS
|
||||
)
|
||||
|
||||
FUNCTIONS = {
|
||||
**parser.Parser.FUNCTIONS,
|
||||
|
|
@ -389,7 +394,7 @@ class MySQL(Dialect):
|
|||
LOCKING_READS_SUPPORTED = True
|
||||
NULL_ORDERING_SUPPORTED = False
|
||||
JOIN_HINTS = False
|
||||
TABLE_HINTS = False
|
||||
TABLE_HINTS = True
|
||||
|
||||
TRANSFORMS = {
|
||||
**generator.Generator.TRANSFORMS,
|
||||
|
|
|
|||
|
|
@ -103,24 +103,15 @@ def _str_to_time_sql(
|
|||
def _ts_or_ds_to_date_sql(self: generator.Generator, expression: exp.TsOrDsToDate) -> str:
|
||||
time_format = self.format_time(expression)
|
||||
if time_format and time_format not in (Presto.TIME_FORMAT, Presto.DATE_FORMAT):
|
||||
return f"CAST({_str_to_time_sql(self, expression)} AS DATE)"
|
||||
return f"CAST(SUBSTR(CAST({self.sql(expression, 'this')} AS VARCHAR), 1, 10) AS DATE)"
|
||||
return exp.cast(_str_to_time_sql(self, expression), "DATE").sql(dialect="presto")
|
||||
return exp.cast(exp.cast(expression.this, "TIMESTAMP"), "DATE").sql(dialect="presto")
|
||||
|
||||
|
||||
def _ts_or_ds_add_sql(self: generator.Generator, expression: exp.TsOrDsAdd) -> str:
|
||||
this = expression.this
|
||||
|
||||
if not isinstance(this, exp.CurrentDate):
|
||||
this = self.func(
|
||||
"DATE_PARSE",
|
||||
self.func(
|
||||
"SUBSTR",
|
||||
this if this.is_string else exp.cast(this, "VARCHAR"),
|
||||
exp.Literal.number(1),
|
||||
exp.Literal.number(10),
|
||||
),
|
||||
Presto.DATE_FORMAT,
|
||||
)
|
||||
this = exp.cast(exp.cast(expression.this, "TIMESTAMP"), "DATE")
|
||||
|
||||
return self.func(
|
||||
"DATE_ADD",
|
||||
|
|
@ -181,6 +172,11 @@ class Presto(Dialect):
|
|||
TIME_MAPPING = MySQL.TIME_MAPPING
|
||||
STRICT_STRING_CONCAT = True
|
||||
|
||||
# https://github.com/trinodb/trino/issues/17
|
||||
# https://github.com/trinodb/trino/issues/12289
|
||||
# https://github.com/prestodb/presto/issues/2863
|
||||
RESOLVES_IDENTIFIERS_AS_UPPERCASE = None
|
||||
|
||||
class Tokenizer(tokens.Tokenizer):
|
||||
KEYWORDS = {
|
||||
**tokens.Tokenizer.KEYWORDS,
|
||||
|
|
|
|||
|
|
@ -14,6 +14,9 @@ def _json_sql(self: Postgres.Generator, expression: exp.JSONExtract | exp.JSONEx
|
|||
|
||||
|
||||
class Redshift(Postgres):
|
||||
# https://docs.aws.amazon.com/redshift/latest/dg/r_names.html
|
||||
RESOLVES_IDENTIFIERS_AS_UPPERCASE = None
|
||||
|
||||
TIME_FORMAT = "'YYYY-MM-DD HH:MI:SS'"
|
||||
TIME_MAPPING = {
|
||||
**Postgres.TIME_MAPPING,
|
||||
|
|
|
|||
|
|
@ -167,6 +167,8 @@ def _parse_convert_timezone(args: t.List) -> exp.Expression:
|
|||
|
||||
|
||||
class Snowflake(Dialect):
|
||||
# https://docs.snowflake.com/en/sql-reference/identifiers-syntax
|
||||
RESOLVES_IDENTIFIERS_AS_UPPERCASE = True
|
||||
NULL_ORDERING = "nulls_are_large"
|
||||
TIME_FORMAT = "'YYYY-MM-DD HH24:MI:SS'"
|
||||
|
||||
|
|
@ -283,11 +285,12 @@ class Snowflake(Dialect):
|
|||
"NCHAR VARYING": TokenType.VARCHAR,
|
||||
"PUT": TokenType.COMMAND,
|
||||
"RENAME": TokenType.REPLACE,
|
||||
"SAMPLE": TokenType.TABLE_SAMPLE,
|
||||
"TIMESTAMP_LTZ": TokenType.TIMESTAMPLTZ,
|
||||
"TIMESTAMP_NTZ": TokenType.TIMESTAMP,
|
||||
"TIMESTAMP_TZ": TokenType.TIMESTAMPTZ,
|
||||
"TIMESTAMPNTZ": TokenType.TIMESTAMP,
|
||||
"SAMPLE": TokenType.TABLE_SAMPLE,
|
||||
"TOP": TokenType.TOP,
|
||||
}
|
||||
|
||||
SINGLE_TOKENS = {
|
||||
|
|
|
|||
|
|
@ -59,6 +59,9 @@ def _transform_create(expression: exp.Expression) -> exp.Expression:
|
|||
|
||||
|
||||
class SQLite(Dialect):
|
||||
# https://sqlite.org/forum/forumpost/5e575586ac5c711b?raw
|
||||
RESOLVES_IDENTIFIERS_AS_UPPERCASE = None
|
||||
|
||||
class Tokenizer(tokens.Tokenizer):
|
||||
IDENTIFIERS = ['"', ("[", "]"), "`"]
|
||||
HEX_STRINGS = [("x'", "'"), ("X'", "'"), ("0x", ""), ("0X", "")]
|
||||
|
|
|
|||
|
|
@ -31,18 +31,19 @@ class Teradata(Dialect):
|
|||
# https://docs.teradata.com/r/Teradata-Database-SQL-Functions-Operators-Expressions-and-Predicates/March-2017/Comparison-Operators-and-Functions/Comparison-Operators/ANSI-Compliance
|
||||
KEYWORDS = {
|
||||
**tokens.Tokenizer.KEYWORDS,
|
||||
"BYTEINT": TokenType.SMALLINT,
|
||||
"SEL": TokenType.SELECT,
|
||||
"INS": TokenType.INSERT,
|
||||
"MOD": TokenType.MOD,
|
||||
"LT": TokenType.LT,
|
||||
"LE": TokenType.LTE,
|
||||
"GT": TokenType.GT,
|
||||
"GE": TokenType.GTE,
|
||||
"^=": TokenType.NEQ,
|
||||
"BYTEINT": TokenType.SMALLINT,
|
||||
"GE": TokenType.GTE,
|
||||
"GT": TokenType.GT,
|
||||
"INS": TokenType.INSERT,
|
||||
"LE": TokenType.LTE,
|
||||
"LT": TokenType.LT,
|
||||
"MOD": TokenType.MOD,
|
||||
"NE": TokenType.NEQ,
|
||||
"NOT=": TokenType.NEQ,
|
||||
"SEL": TokenType.SELECT,
|
||||
"ST_GEOMETRY": TokenType.GEOMETRY,
|
||||
"TOP": TokenType.TOP,
|
||||
}
|
||||
|
||||
# Teradata does not support % as a modulo operator
|
||||
|
|
|
|||
|
|
@ -1301,7 +1301,14 @@ class Constraint(Expression):
|
|||
|
||||
|
||||
class Delete(Expression):
|
||||
arg_types = {"with": False, "this": False, "using": False, "where": False, "returning": False}
|
||||
arg_types = {
|
||||
"with": False,
|
||||
"this": False,
|
||||
"using": False,
|
||||
"where": False,
|
||||
"returning": False,
|
||||
"limit": False,
|
||||
}
|
||||
|
||||
def delete(
|
||||
self,
|
||||
|
|
@ -1844,6 +1851,10 @@ class CollateProperty(Property):
|
|||
arg_types = {"this": True}
|
||||
|
||||
|
||||
class CopyGrantsProperty(Property):
|
||||
arg_types = {}
|
||||
|
||||
|
||||
class DataBlocksizeProperty(Property):
|
||||
arg_types = {
|
||||
"size": False,
|
||||
|
|
@ -2245,6 +2256,16 @@ QUERY_MODIFIERS = {
|
|||
}
|
||||
|
||||
|
||||
# https://learn.microsoft.com/en-us/sql/t-sql/queries/hints-transact-sql-table?view=sql-server-ver16
|
||||
class WithTableHint(Expression):
|
||||
arg_types = {"expressions": True}
|
||||
|
||||
|
||||
# https://dev.mysql.com/doc/refman/8.0/en/index-hints.html
|
||||
class IndexTableHint(Expression):
|
||||
arg_types = {"this": True, "expressions": False, "target": False}
|
||||
|
||||
|
||||
class Table(Expression):
|
||||
arg_types = {
|
||||
"this": True,
|
||||
|
|
@ -2402,6 +2423,7 @@ class Update(Expression):
|
|||
"from": False,
|
||||
"where": False,
|
||||
"returning": False,
|
||||
"limit": False,
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -2434,8 +2456,6 @@ class Select(Subqueryable):
|
|||
"expressions": False,
|
||||
"hint": False,
|
||||
"distinct": False,
|
||||
"struct": False, # https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#return_query_results_as_a_value_table
|
||||
"value": False,
|
||||
"into": False,
|
||||
"from": False,
|
||||
**QUERY_MODIFIERS,
|
||||
|
|
@ -3223,15 +3243,15 @@ class Star(Expression):
|
|||
return self.name
|
||||
|
||||
|
||||
class Parameter(Expression):
|
||||
class Parameter(Condition):
|
||||
arg_types = {"this": True, "wrapped": False}
|
||||
|
||||
|
||||
class SessionParameter(Expression):
|
||||
class SessionParameter(Condition):
|
||||
arg_types = {"this": True, "kind": False}
|
||||
|
||||
|
||||
class Placeholder(Expression):
|
||||
class Placeholder(Condition):
|
||||
arg_types = {"this": False, "kind": False}
|
||||
|
||||
|
||||
|
|
@ -3333,6 +3353,7 @@ class DataType(Expression):
|
|||
UINT128 = auto()
|
||||
UINT256 = auto()
|
||||
UNIQUEIDENTIFIER = auto()
|
||||
USERDEFINED = "USER-DEFINED"
|
||||
UUID = auto()
|
||||
VARBINARY = auto()
|
||||
VARCHAR = auto()
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ import typing as t
|
|||
|
||||
from sqlglot import exp
|
||||
from sqlglot.errors import ErrorLevel, UnsupportedError, concat_messages
|
||||
from sqlglot.helper import apply_index_offset, csv, seq_get, should_identify
|
||||
from sqlglot.helper import apply_index_offset, csv, seq_get
|
||||
from sqlglot.time import format_time
|
||||
from sqlglot.tokens import TokenType
|
||||
|
||||
|
|
@ -56,39 +56,40 @@ class Generator:
|
|||
exp.TsOrDsAdd: lambda self, e: self.func(
|
||||
"TS_OR_DS_ADD", e.this, e.expression, exp.Literal.string(e.text("unit"))
|
||||
),
|
||||
exp.VarMap: lambda self, e: self.func("MAP", e.args["keys"], e.args["values"]),
|
||||
exp.CaseSpecificColumnConstraint: lambda self, e: f"{'NOT ' if e.args.get('not_') else ''}CASESPECIFIC",
|
||||
exp.CharacterSetColumnConstraint: lambda self, e: f"CHARACTER SET {self.sql(e, 'this')}",
|
||||
exp.CharacterSetProperty: lambda self, e: f"{'DEFAULT ' if e.args.get('default') else ''}CHARACTER SET={self.sql(e, 'this')}",
|
||||
exp.CheckColumnConstraint: lambda self, e: f"CHECK ({self.sql(e, 'this')})",
|
||||
exp.CollateColumnConstraint: lambda self, e: f"COLLATE {self.sql(e, 'this')}",
|
||||
exp.CopyGrantsProperty: lambda self, e: "COPY GRANTS",
|
||||
exp.CommentColumnConstraint: lambda self, e: f"COMMENT {self.sql(e, 'this')}",
|
||||
exp.DateFormatColumnConstraint: lambda self, e: f"FORMAT {self.sql(e, 'this')}",
|
||||
exp.DefaultColumnConstraint: lambda self, e: f"DEFAULT {self.sql(e, 'this')}",
|
||||
exp.EncodeColumnConstraint: lambda self, e: f"ENCODE {self.sql(e, 'this')}",
|
||||
exp.ExecuteAsProperty: lambda self, e: self.naked_property(e),
|
||||
exp.ExternalProperty: lambda self, e: "EXTERNAL",
|
||||
exp.InlineLengthColumnConstraint: lambda self, e: f"INLINE LENGTH {self.sql(e, 'this')}",
|
||||
exp.LanguageProperty: lambda self, e: self.naked_property(e),
|
||||
exp.LocationProperty: lambda self, e: self.naked_property(e),
|
||||
exp.LogProperty: lambda self, e: f"{'NO ' if e.args.get('no') else ''}LOG",
|
||||
exp.MaterializedProperty: lambda self, e: "MATERIALIZED",
|
||||
exp.NoPrimaryIndexProperty: lambda self, e: "NO PRIMARY INDEX",
|
||||
exp.OnCommitProperty: lambda self, e: f"ON COMMIT {'DELETE' if e.args.get('delete') else 'PRESERVE'} ROWS",
|
||||
exp.OnUpdateColumnConstraint: lambda self, e: f"ON UPDATE {self.sql(e, 'this')}",
|
||||
exp.PathColumnConstraint: lambda self, e: f"PATH {self.sql(e, 'this')}",
|
||||
exp.ReturnsProperty: lambda self, e: self.naked_property(e),
|
||||
exp.SetProperty: lambda self, e: f"{'MULTI' if e.args.get('multi') else ''}SET",
|
||||
exp.SettingsProperty: lambda self, e: f"SETTINGS{self.seg('')}{(self.expressions(e))}",
|
||||
exp.SqlSecurityProperty: lambda self, e: f"SQL SECURITY {'DEFINER' if e.args.get('definer') else 'INVOKER'}",
|
||||
exp.StabilityProperty: lambda self, e: e.name,
|
||||
exp.TemporaryProperty: lambda self, e: f"TEMPORARY",
|
||||
exp.ToTableProperty: lambda self, e: f"TO {self.sql(e.this)}",
|
||||
exp.TransientProperty: lambda self, e: "TRANSIENT",
|
||||
exp.StabilityProperty: lambda self, e: e.name,
|
||||
exp.TitleColumnConstraint: lambda self, e: f"TITLE {self.sql(e, 'this')}",
|
||||
exp.UppercaseColumnConstraint: lambda self, e: f"UPPERCASE",
|
||||
exp.VarMap: lambda self, e: self.func("MAP", e.args["keys"], e.args["values"]),
|
||||
exp.VolatileProperty: lambda self, e: "VOLATILE",
|
||||
exp.WithJournalTableProperty: lambda self, e: f"WITH JOURNAL TABLE={self.sql(e, 'this')}",
|
||||
exp.CaseSpecificColumnConstraint: lambda self, e: f"{'NOT ' if e.args.get('not_') else ''}CASESPECIFIC",
|
||||
exp.CharacterSetColumnConstraint: lambda self, e: f"CHARACTER SET {self.sql(e, 'this')}",
|
||||
exp.DateFormatColumnConstraint: lambda self, e: f"FORMAT {self.sql(e, 'this')}",
|
||||
exp.OnUpdateColumnConstraint: lambda self, e: f"ON UPDATE {self.sql(e, 'this')}",
|
||||
exp.UppercaseColumnConstraint: lambda self, e: f"UPPERCASE",
|
||||
exp.TitleColumnConstraint: lambda self, e: f"TITLE {self.sql(e, 'this')}",
|
||||
exp.PathColumnConstraint: lambda self, e: f"PATH {self.sql(e, 'this')}",
|
||||
exp.CheckColumnConstraint: lambda self, e: f"CHECK ({self.sql(e, 'this')})",
|
||||
exp.CommentColumnConstraint: lambda self, e: f"COMMENT {self.sql(e, 'this')}",
|
||||
exp.CollateColumnConstraint: lambda self, e: f"COLLATE {self.sql(e, 'this')}",
|
||||
exp.EncodeColumnConstraint: lambda self, e: f"ENCODE {self.sql(e, 'this')}",
|
||||
exp.DefaultColumnConstraint: lambda self, e: f"DEFAULT {self.sql(e, 'this')}",
|
||||
exp.InlineLengthColumnConstraint: lambda self, e: f"INLINE LENGTH {self.sql(e, 'this')}",
|
||||
}
|
||||
|
||||
# Whether or not null ordering is supported in order by
|
||||
|
|
@ -142,6 +143,9 @@ class Generator:
|
|||
# Whether or not comparing against booleans (e.g. x IS TRUE) is supported
|
||||
IS_BOOL_ALLOWED = True
|
||||
|
||||
# https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax
|
||||
SELECT_KINDS: t.Tuple[str, ...] = ("STRUCT", "VALUE")
|
||||
|
||||
TYPE_MAPPING = {
|
||||
exp.DataType.Type.NCHAR: "CHAR",
|
||||
exp.DataType.Type.NVARCHAR: "VARCHAR",
|
||||
|
|
@ -182,6 +186,7 @@ class Generator:
|
|||
exp.CharacterSetProperty: exp.Properties.Location.POST_SCHEMA,
|
||||
exp.ChecksumProperty: exp.Properties.Location.POST_NAME,
|
||||
exp.CollateProperty: exp.Properties.Location.POST_SCHEMA,
|
||||
exp.CopyGrantsProperty: exp.Properties.Location.POST_SCHEMA,
|
||||
exp.Cluster: exp.Properties.Location.POST_SCHEMA,
|
||||
exp.DataBlocksizeProperty: exp.Properties.Location.POST_NAME,
|
||||
exp.DefinerProperty: exp.Properties.Location.POST_CREATE,
|
||||
|
|
@ -263,6 +268,8 @@ class Generator:
|
|||
NORMALIZE_FUNCTIONS: bool | str = "upper"
|
||||
NULL_ORDERING = "nulls_are_small"
|
||||
|
||||
can_identify: t.Callable[[str, str | bool], bool]
|
||||
|
||||
# Delimiters for quotes, identifiers and the corresponding escape characters
|
||||
QUOTE_START = "'"
|
||||
QUOTE_END = "'"
|
||||
|
|
@ -771,9 +778,11 @@ class Generator:
|
|||
return this
|
||||
|
||||
def rawstring_sql(self, expression: exp.RawString) -> str:
|
||||
string = expression.this
|
||||
if self.RAW_START:
|
||||
return f"{self.RAW_START}{expression.name}{self.RAW_END}"
|
||||
return self.sql(exp.Literal.string(expression.name.replace("\\", "\\\\")))
|
||||
return f"{self.RAW_START}{self.escape_str(expression.this)}{self.RAW_END}"
|
||||
string = self.escape_str(string.replace("\\", "\\\\"))
|
||||
return f"{self.QUOTE_START}{string}{self.QUOTE_END}"
|
||||
|
||||
def datatypesize_sql(self, expression: exp.DataTypeSize) -> str:
|
||||
this = self.sql(expression, "this")
|
||||
|
|
@ -815,7 +824,8 @@ class Generator:
|
|||
)
|
||||
where_sql = self.sql(expression, "where")
|
||||
returning = self.sql(expression, "returning")
|
||||
sql = f"DELETE{this}{using_sql}{where_sql}{returning}"
|
||||
limit = self.sql(expression, "limit")
|
||||
sql = f"DELETE{this}{using_sql}{where_sql}{returning}{limit}"
|
||||
return self.prepend_ctes(expression, sql)
|
||||
|
||||
def drop_sql(self, expression: exp.Drop) -> str:
|
||||
|
|
@ -883,7 +893,7 @@ class Generator:
|
|||
text = text.replace(self.IDENTIFIER_END, self._escaped_identifier_end)
|
||||
if (
|
||||
expression.quoted
|
||||
or should_identify(text, self.identify)
|
||||
or self.can_identify(text, self.identify)
|
||||
or lower in self.RESERVED_KEYWORDS
|
||||
or (not self.IDENTIFIERS_CAN_START_WITH_DIGIT and text[:1].isdigit())
|
||||
):
|
||||
|
|
@ -1157,6 +1167,15 @@ class Generator:
|
|||
null = f" NULL DEFINED AS {null}" if null else ""
|
||||
return f"ROW FORMAT DELIMITED{fields}{escaped}{items}{keys}{lines}{null}"
|
||||
|
||||
def withtablehint_sql(self, expression: exp.WithTableHint) -> str:
|
||||
return f"WITH ({self.expressions(expression, flat=True)})"
|
||||
|
||||
def indextablehint_sql(self, expression: exp.IndexTableHint) -> str:
|
||||
this = f"{self.sql(expression, 'this')} INDEX"
|
||||
target = self.sql(expression, "target")
|
||||
target = f" FOR {target}" if target else ""
|
||||
return f"{this}{target} ({self.expressions(expression, flat=True)})"
|
||||
|
||||
def table_sql(self, expression: exp.Table, sep: str = " AS ") -> str:
|
||||
table = ".".join(
|
||||
part
|
||||
|
|
@ -1170,8 +1189,8 @@ class Generator:
|
|||
|
||||
alias = self.sql(expression, "alias")
|
||||
alias = f"{sep}{alias}" if alias else ""
|
||||
hints = self.expressions(expression, key="hints", flat=True)
|
||||
hints = f" WITH ({hints})" if hints and self.TABLE_HINTS else ""
|
||||
hints = self.expressions(expression, key="hints", sep=" ")
|
||||
hints = f" {hints}" if hints and self.TABLE_HINTS else ""
|
||||
pivots = self.expressions(expression, key="pivots", sep=" ", flat=True)
|
||||
pivots = f" {pivots}" if pivots else ""
|
||||
joins = self.expressions(expression, key="joins", sep="")
|
||||
|
|
@ -1238,7 +1257,8 @@ class Generator:
|
|||
from_sql = self.sql(expression, "from")
|
||||
where_sql = self.sql(expression, "where")
|
||||
returning = self.sql(expression, "returning")
|
||||
sql = f"UPDATE {this} SET {set_sql}{from_sql}{where_sql}{returning}"
|
||||
limit = self.sql(expression, "limit")
|
||||
sql = f"UPDATE {this} SET {set_sql}{from_sql}{where_sql}{returning}{limit}"
|
||||
return self.prepend_ctes(expression, sql)
|
||||
|
||||
def values_sql(self, expression: exp.Values) -> str:
|
||||
|
|
@ -1413,10 +1433,13 @@ class Generator:
|
|||
def literal_sql(self, expression: exp.Literal) -> str:
|
||||
text = expression.this or ""
|
||||
if expression.is_string:
|
||||
text = f"{self.QUOTE_START}{self.escape_str(text)}{self.QUOTE_END}"
|
||||
return text
|
||||
|
||||
def escape_str(self, text: str) -> str:
|
||||
text = text.replace(self.QUOTE_END, self._escaped_quote_end)
|
||||
if self.pretty:
|
||||
text = text.replace("\n", self.SENTINEL_LINE_BREAK)
|
||||
text = f"{self.QUOTE_START}{text}{self.QUOTE_END}"
|
||||
return text
|
||||
|
||||
def loaddata_sql(self, expression: exp.LoadData) -> str:
|
||||
|
|
@ -1565,9 +1588,30 @@ class Generator:
|
|||
hint = self.sql(expression, "hint")
|
||||
distinct = self.sql(expression, "distinct")
|
||||
distinct = f" {distinct}" if distinct else ""
|
||||
kind = expression.args.get("kind")
|
||||
kind = f" AS {kind}" if kind else ""
|
||||
kind = self.sql(expression, "kind").upper()
|
||||
expressions = self.expressions(expression)
|
||||
|
||||
if kind:
|
||||
if kind in self.SELECT_KINDS:
|
||||
kind = f" AS {kind}"
|
||||
else:
|
||||
if kind == "STRUCT":
|
||||
expressions = self.expressions(
|
||||
sqls=[
|
||||
self.sql(
|
||||
exp.Struct(
|
||||
expressions=[
|
||||
exp.column(e.output_name).eq(
|
||||
e.this if isinstance(e, exp.Alias) else e
|
||||
)
|
||||
for e in expression.expressions
|
||||
]
|
||||
)
|
||||
)
|
||||
]
|
||||
)
|
||||
kind = ""
|
||||
|
||||
expressions = f"{self.sep()}{expressions}" if expressions else expressions
|
||||
sql = self.query_modifiers(
|
||||
expression,
|
||||
|
|
|
|||
|
|
@ -14,7 +14,6 @@ from itertools import count
|
|||
if t.TYPE_CHECKING:
|
||||
from sqlglot import exp
|
||||
from sqlglot._typing import E, T
|
||||
from sqlglot.dialects.dialect import DialectType
|
||||
from sqlglot.expressions import Expression
|
||||
|
||||
CAMEL_CASE_PATTERN = re.compile("(?<!^)(?=[A-Z])")
|
||||
|
|
@ -23,7 +22,12 @@ logger = logging.getLogger("sqlglot")
|
|||
|
||||
|
||||
class AutoName(Enum):
|
||||
"""This is used for creating enum classes where `auto()` is the string form of the corresponding value's name."""
|
||||
"""
|
||||
This is used for creating Enum classes where `auto()` is the string form
|
||||
of the corresponding enum's identifier (e.g. FOO.value results in "FOO").
|
||||
|
||||
Reference: https://docs.python.org/3/howto/enum.html#using-automatic-values
|
||||
"""
|
||||
|
||||
def _generate_next_value_(name, _start, _count, _last_values):
|
||||
return name
|
||||
|
|
@ -52,7 +56,7 @@ def ensure_list(value):
|
|||
Ensures that a value is a list, otherwise casts or wraps it into one.
|
||||
|
||||
Args:
|
||||
value: the value of interest.
|
||||
value: The value of interest.
|
||||
|
||||
Returns:
|
||||
The value cast as a list if it's a list or a tuple, or else the value wrapped in a list.
|
||||
|
|
@ -80,7 +84,7 @@ def ensure_collection(value):
|
|||
Ensures that a value is a collection (excluding `str` and `bytes`), otherwise wraps it into a list.
|
||||
|
||||
Args:
|
||||
value: the value of interest.
|
||||
value: The value of interest.
|
||||
|
||||
Returns:
|
||||
The value if it's a collection, or else the value wrapped in a list.
|
||||
|
|
@ -97,8 +101,8 @@ def csv(*args: str, sep: str = ", ") -> str:
|
|||
Formats any number of string arguments as CSV.
|
||||
|
||||
Args:
|
||||
args: the string arguments to format.
|
||||
sep: the argument separator.
|
||||
args: The string arguments to format.
|
||||
sep: The argument separator.
|
||||
|
||||
Returns:
|
||||
The arguments formatted as a CSV string.
|
||||
|
|
@ -115,9 +119,9 @@ def subclasses(
|
|||
Returns all subclasses for a collection of classes, possibly excluding some of them.
|
||||
|
||||
Args:
|
||||
module_name: the name of the module to search for subclasses in.
|
||||
classes: class(es) we want to find the subclasses of.
|
||||
exclude: class(es) we want to exclude from the returned list.
|
||||
module_name: The name of the module to search for subclasses in.
|
||||
classes: Class(es) we want to find the subclasses of.
|
||||
exclude: Class(es) we want to exclude from the returned list.
|
||||
|
||||
Returns:
|
||||
The target subclasses.
|
||||
|
|
@ -140,13 +144,13 @@ def apply_index_offset(
|
|||
Applies an offset to a given integer literal expression.
|
||||
|
||||
Args:
|
||||
this: the target of the index
|
||||
expressions: the expression the offset will be applied to, wrapped in a list.
|
||||
offset: the offset that will be applied.
|
||||
this: The target of the index.
|
||||
expressions: The expression the offset will be applied to, wrapped in a list.
|
||||
offset: The offset that will be applied.
|
||||
|
||||
Returns:
|
||||
The original expression with the offset applied to it, wrapped in a list. If the provided
|
||||
`expressions` argument contains more than one expressions, it's returned unaffected.
|
||||
`expressions` argument contains more than one expression, it's returned unaffected.
|
||||
"""
|
||||
if not offset or len(expressions) != 1:
|
||||
return expressions
|
||||
|
|
@ -189,8 +193,8 @@ def while_changing(expression: Expression, func: t.Callable[[Expression], E]) ->
|
|||
Applies a transformation to a given expression until a fix point is reached.
|
||||
|
||||
Args:
|
||||
expression: the expression to be transformed.
|
||||
func: the transformation to be applied.
|
||||
expression: The expression to be transformed.
|
||||
func: The transformation to be applied.
|
||||
|
||||
Returns:
|
||||
The transformed expression.
|
||||
|
|
@ -198,6 +202,7 @@ def while_changing(expression: Expression, func: t.Callable[[Expression], E]) ->
|
|||
while True:
|
||||
for n, *_ in reversed(tuple(expression.walk())):
|
||||
n._hash = hash(n)
|
||||
|
||||
start = hash(expression)
|
||||
expression = func(expression)
|
||||
|
||||
|
|
@ -205,6 +210,7 @@ def while_changing(expression: Expression, func: t.Callable[[Expression], E]) ->
|
|||
n._hash = None
|
||||
if start == hash(expression):
|
||||
break
|
||||
|
||||
return expression
|
||||
|
||||
|
||||
|
|
@ -213,7 +219,7 @@ def tsort(dag: t.Dict[T, t.Set[T]]) -> t.List[T]:
|
|||
Sorts a given directed acyclic graph in topological order.
|
||||
|
||||
Args:
|
||||
dag: the graph to be sorted.
|
||||
dag: The graph to be sorted.
|
||||
|
||||
Returns:
|
||||
A list that contains all of the graph's nodes in topological order.
|
||||
|
|
@ -261,7 +267,7 @@ def csv_reader(read_csv: exp.ReadCSV) -> t.Any:
|
|||
Returns a csv reader given the expression `READ_CSV(name, ['delimiter', '|', ...])`.
|
||||
|
||||
Args:
|
||||
read_csv: a `ReadCSV` function call
|
||||
read_csv: A `ReadCSV` function call.
|
||||
|
||||
Yields:
|
||||
A python csv reader.
|
||||
|
|
@ -288,8 +294,8 @@ def find_new_name(taken: t.Collection[str], base: str) -> str:
|
|||
Searches for a new name.
|
||||
|
||||
Args:
|
||||
taken: a collection of taken names.
|
||||
base: base name to alter.
|
||||
taken: A collection of taken names.
|
||||
base: Base name to alter.
|
||||
|
||||
Returns:
|
||||
The new, available name.
|
||||
|
|
@ -327,10 +333,10 @@ def split_num_words(
|
|||
Perform a split on a value and return N words as a result with `None` used for words that don't exist.
|
||||
|
||||
Args:
|
||||
value: the value to be split.
|
||||
sep: the value to use to split on.
|
||||
min_num_words: the minimum number of words that are going to be in the result.
|
||||
fill_from_start: indicates that if `None` values should be inserted at the start or end of the list.
|
||||
value: The value to be split.
|
||||
sep: The value to use to split on.
|
||||
min_num_words: The minimum number of words that are going to be in the result.
|
||||
fill_from_start: Indicates that if `None` values should be inserted at the start or end of the list.
|
||||
|
||||
Examples:
|
||||
>>> split_num_words("db.table", ".", 3)
|
||||
|
|
@ -360,7 +366,7 @@ def is_iterable(value: t.Any) -> bool:
|
|||
False
|
||||
|
||||
Args:
|
||||
value: the value to check if it is an iterable.
|
||||
value: The value to check if it is an iterable.
|
||||
|
||||
Returns:
|
||||
A `bool` value indicating if it is an iterable.
|
||||
|
|
@ -380,7 +386,7 @@ def flatten(values: t.Iterable[t.Iterable[t.Any] | t.Any]) -> t.Iterator[t.Any]:
|
|||
[1, 2, 3]
|
||||
|
||||
Args:
|
||||
values: the value to be flattened.
|
||||
values: The value to be flattened.
|
||||
|
||||
Yields:
|
||||
Non-iterable elements in `values`.
|
||||
|
|
@ -396,7 +402,7 @@ def dict_depth(d: t.Dict) -> int:
|
|||
"""
|
||||
Get the nesting depth of a dictionary.
|
||||
|
||||
For example:
|
||||
Example:
|
||||
>>> dict_depth(None)
|
||||
0
|
||||
>>> dict_depth({})
|
||||
|
|
@ -407,12 +413,6 @@ def dict_depth(d: t.Dict) -> int:
|
|||
2
|
||||
>>> dict_depth({"a": {"b": {}}})
|
||||
3
|
||||
|
||||
Args:
|
||||
d (dict): dictionary
|
||||
|
||||
Returns:
|
||||
int: depth
|
||||
"""
|
||||
try:
|
||||
return 1 + dict_depth(next(iter(d.values())))
|
||||
|
|
@ -425,36 +425,5 @@ def dict_depth(d: t.Dict) -> int:
|
|||
|
||||
|
||||
def first(it: t.Iterable[T]) -> T:
|
||||
"""Returns the first element from an iterable.
|
||||
|
||||
Useful for sets.
|
||||
"""
|
||||
"""Returns the first element from an iterable (useful for sets)."""
|
||||
return next(i for i in it)
|
||||
|
||||
|
||||
def case_sensitive(text: str, dialect: DialectType) -> bool:
|
||||
"""Checks if text contains any case sensitive characters depending on dialect."""
|
||||
from sqlglot.dialects.dialect import RESOLVES_IDENTIFIERS_AS_UPPERCASE
|
||||
|
||||
unsafe = str.islower if dialect in RESOLVES_IDENTIFIERS_AS_UPPERCASE else str.isupper
|
||||
return any(unsafe(char) for char in text)
|
||||
|
||||
|
||||
def should_identify(text: str, identify: str | bool, dialect: DialectType = None) -> bool:
|
||||
"""Checks if text should be identified given an identify option.
|
||||
|
||||
Args:
|
||||
text: the text to check.
|
||||
identify:
|
||||
"always" or `True`: always returns true.
|
||||
"safe": true if there is no uppercase or lowercase character in `text`, depending on `dialect`.
|
||||
dialect: the dialect to use in order to decide whether a text should be identified.
|
||||
|
||||
Returns:
|
||||
Whether or not a string should be identified.
|
||||
"""
|
||||
if identify is True or identify == "always":
|
||||
return True
|
||||
if identify == "safe":
|
||||
return not case_sensitive(text, dialect)
|
||||
return False
|
||||
|
|
|
|||
|
|
@ -187,7 +187,7 @@ class TypeAnnotator(metaclass=_TypeAnnotator):
|
|||
},
|
||||
}
|
||||
|
||||
ANNOTATORS = {
|
||||
ANNOTATORS: t.Dict = {
|
||||
**{
|
||||
expr_type: lambda self, e: self._annotate_unary(e)
|
||||
for expr_type in subclasses(exp.__name__, (exp.Unary, exp.Alias))
|
||||
|
|
|
|||
|
|
@ -1,12 +1,15 @@
|
|||
from sqlglot import exp
|
||||
from sqlglot._typing import E
|
||||
from sqlglot.dialects.dialect import RESOLVES_IDENTIFIERS_AS_UPPERCASE, DialectType
|
||||
from sqlglot.dialects.dialect import Dialect, DialectType
|
||||
|
||||
|
||||
def normalize_identifiers(expression: E, dialect: DialectType = None) -> E:
|
||||
"""
|
||||
Normalize all unquoted identifiers to either lower or upper case, depending on
|
||||
the dialect. This essentially makes those identifiers case-insensitive.
|
||||
Normalize all unquoted identifiers to either lower or upper case, depending
|
||||
on the dialect. This essentially makes those identifiers case-insensitive.
|
||||
|
||||
Note:
|
||||
Some dialects (e.g. BigQuery) treat identifiers as case-insensitive even
|
||||
when they're quoted, so in these cases all identifiers are normalized.
|
||||
|
||||
Example:
|
||||
>>> import sqlglot
|
||||
|
|
@ -21,16 +24,4 @@ def normalize_identifiers(expression: E, dialect: DialectType = None) -> E:
|
|||
Returns:
|
||||
The transformed expression.
|
||||
"""
|
||||
return expression.transform(_normalize, dialect, copy=False)
|
||||
|
||||
|
||||
def _normalize(node: exp.Expression, dialect: DialectType = None) -> exp.Expression:
|
||||
if isinstance(node, exp.Identifier) and not node.quoted:
|
||||
node.set(
|
||||
"this",
|
||||
node.this.upper()
|
||||
if dialect in RESOLVES_IDENTIFIERS_AS_UPPERCASE
|
||||
else node.this.lower(),
|
||||
)
|
||||
|
||||
return node
|
||||
return expression.transform(Dialect.get_or_raise(dialect).normalize_identifier, copy=False)
|
||||
|
|
|
|||
|
|
@ -5,9 +5,9 @@ import typing as t
|
|||
|
||||
from sqlglot import alias, exp
|
||||
from sqlglot._typing import E
|
||||
from sqlglot.dialects.dialect import DialectType
|
||||
from sqlglot.dialects.dialect import Dialect, DialectType
|
||||
from sqlglot.errors import OptimizeError
|
||||
from sqlglot.helper import case_sensitive, seq_get
|
||||
from sqlglot.helper import seq_get
|
||||
from sqlglot.optimizer.scope import Scope, traverse_scope, walk_in_scope
|
||||
from sqlglot.schema import Schema, ensure_schema
|
||||
|
||||
|
|
@ -417,19 +417,9 @@ def _qualify_outputs(scope):
|
|||
|
||||
def quote_identifiers(expression: E, dialect: DialectType = None, identify: bool = True) -> E:
|
||||
"""Makes sure all identifiers that need to be quoted are quoted."""
|
||||
|
||||
def _quote(expression: E) -> E:
|
||||
if isinstance(expression, exp.Identifier):
|
||||
name = expression.this
|
||||
expression.set(
|
||||
"quoted",
|
||||
identify
|
||||
or case_sensitive(name, dialect=dialect)
|
||||
or not exp.SAFE_IDENTIFIER_RE.match(name),
|
||||
return expression.transform(
|
||||
Dialect.get_or_raise(dialect).quote_identifier, identify=identify, copy=False
|
||||
)
|
||||
return expression
|
||||
|
||||
return expression.transform(_quote, copy=False)
|
||||
|
||||
|
||||
class Resolver:
|
||||
|
|
|
|||
|
|
@ -408,9 +408,14 @@ def remove_where_true(expression):
|
|||
if always_true(where.this):
|
||||
where.parent.set("where", None)
|
||||
for join in expression.find_all(exp.Join):
|
||||
if always_true(join.args.get("on")):
|
||||
join.set("kind", "CROSS")
|
||||
if (
|
||||
always_true(join.args.get("on"))
|
||||
and not join.args.get("using")
|
||||
and not join.args.get("method")
|
||||
):
|
||||
join.set("on", None)
|
||||
join.set("side", None)
|
||||
join.set("kind", "CROSS")
|
||||
|
||||
|
||||
def always_true(expression):
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ from sqlglot.errors import ErrorLevel, ParseError, concat_messages, merge_errors
|
|||
from sqlglot.helper import apply_index_offset, ensure_list, seq_get
|
||||
from sqlglot.time import format_time
|
||||
from sqlglot.tokens import Token, Tokenizer, TokenType
|
||||
from sqlglot.trie import in_trie, new_trie
|
||||
from sqlglot.trie import TrieResult, in_trie, new_trie
|
||||
|
||||
if t.TYPE_CHECKING:
|
||||
from sqlglot._typing import E
|
||||
|
|
@ -177,6 +177,7 @@ class Parser(metaclass=_Parser):
|
|||
TokenType.BIGSERIAL,
|
||||
TokenType.XML,
|
||||
TokenType.UNIQUEIDENTIFIER,
|
||||
TokenType.USERDEFINED,
|
||||
TokenType.MONEY,
|
||||
TokenType.SMALLMONEY,
|
||||
TokenType.ROWVERSION,
|
||||
|
|
@ -465,7 +466,7 @@ class Parser(metaclass=_Parser):
|
|||
}
|
||||
|
||||
EXPRESSION_PARSERS = {
|
||||
exp.Cluster: lambda self: self._parse_sort(exp.Cluster, "CLUSTER", "BY"),
|
||||
exp.Cluster: lambda self: self._parse_sort(exp.Cluster, TokenType.CLUSTER_BY),
|
||||
exp.Column: lambda self: self._parse_column(),
|
||||
exp.Condition: lambda self: self._parse_conjunction(),
|
||||
exp.DataType: lambda self: self._parse_types(),
|
||||
|
|
@ -484,7 +485,7 @@ class Parser(metaclass=_Parser):
|
|||
exp.Properties: lambda self: self._parse_properties(),
|
||||
exp.Qualify: lambda self: self._parse_qualify(),
|
||||
exp.Returning: lambda self: self._parse_returning(),
|
||||
exp.Sort: lambda self: self._parse_sort(exp.Sort, "SORT", "BY"),
|
||||
exp.Sort: lambda self: self._parse_sort(exp.Sort, TokenType.SORT_BY),
|
||||
exp.Table: lambda self: self._parse_table_parts(),
|
||||
exp.TableAlias: lambda self: self._parse_table_alias(),
|
||||
exp.Where: lambda self: self._parse_where(),
|
||||
|
|
@ -540,8 +541,7 @@ class Parser(metaclass=_Parser):
|
|||
exp.Literal, this=token.text, is_string=False
|
||||
),
|
||||
TokenType.STAR: lambda self, _: self.expression(
|
||||
exp.Star,
|
||||
**{"except": self._parse_except(), "replace": self._parse_replace()},
|
||||
exp.Star, **{"except": self._parse_except(), "replace": self._parse_replace()}
|
||||
),
|
||||
TokenType.NULL: lambda self, _: self.expression(exp.Null),
|
||||
TokenType.TRUE: lambda self, _: self.expression(exp.Boolean, this=True),
|
||||
|
|
@ -584,9 +584,10 @@ class Parser(metaclass=_Parser):
|
|||
"BLOCKCOMPRESSION": lambda self: self._parse_blockcompression(),
|
||||
"CHARACTER SET": lambda self: self._parse_character_set(),
|
||||
"CHECKSUM": lambda self: self._parse_checksum(),
|
||||
"CLUSTER": lambda self: self._parse_cluster(),
|
||||
"CLUSTER BY": lambda self: self._parse_cluster(),
|
||||
"COLLATE": lambda self: self._parse_property_assignment(exp.CollateProperty),
|
||||
"COMMENT": lambda self: self._parse_property_assignment(exp.SchemaCommentProperty),
|
||||
"COPY": lambda self: self._parse_copy_property(),
|
||||
"DATABLOCKSIZE": lambda self, **kwargs: self._parse_datablocksize(**kwargs),
|
||||
"DEFINER": lambda self: self._parse_definer(),
|
||||
"DETERMINISTIC": lambda self: self.expression(
|
||||
|
|
@ -780,6 +781,8 @@ class Parser(metaclass=_Parser):
|
|||
|
||||
CLONE_KINDS = {"TIMESTAMP", "OFFSET", "STATEMENT"}
|
||||
|
||||
TABLE_INDEX_HINT_TOKENS = {TokenType.FORCE, TokenType.IGNORE, TokenType.USE}
|
||||
|
||||
WINDOW_ALIAS_TOKENS = ID_VAR_TOKENS - {TokenType.ROWS}
|
||||
WINDOW_BEFORE_PAREN_TOKENS = {TokenType.OVER}
|
||||
WINDOW_SIDES = {"FOLLOWING", "PRECEDING"}
|
||||
|
|
@ -788,7 +791,8 @@ class Parser(metaclass=_Parser):
|
|||
|
||||
STRICT_CAST = True
|
||||
|
||||
CONCAT_NULL_OUTPUTS_STRING = False # A NULL arg in CONCAT yields NULL by default
|
||||
# A NULL arg in CONCAT yields NULL by default
|
||||
CONCAT_NULL_OUTPUTS_STRING = False
|
||||
|
||||
CONVERT_TYPE_FIRST = False
|
||||
|
||||
|
|
@ -1423,11 +1427,14 @@ class Parser(metaclass=_Parser):
|
|||
return self.expression(exp.ChecksumProperty, on=on, default=self._match(TokenType.DEFAULT))
|
||||
|
||||
def _parse_cluster(self) -> t.Optional[exp.Cluster]:
|
||||
if not self._match_text_seq("BY"):
|
||||
return self.expression(exp.Cluster, expressions=self._parse_csv(self._parse_ordered))
|
||||
|
||||
def _parse_copy_property(self) -> t.Optional[exp.CopyGrantsProperty]:
|
||||
if not self._match_text_seq("GRANTS"):
|
||||
self._retreat(self._index - 1)
|
||||
return None
|
||||
|
||||
return self.expression(exp.Cluster, expressions=self._parse_csv(self._parse_ordered))
|
||||
return self.expression(exp.CopyGrantsProperty)
|
||||
|
||||
def _parse_freespace(self) -> exp.FreespaceProperty:
|
||||
self._match(TokenType.EQ)
|
||||
|
|
@ -1779,6 +1786,7 @@ class Parser(metaclass=_Parser):
|
|||
using=self._parse_csv(lambda: self._match(TokenType.USING) and self._parse_table()),
|
||||
where=self._parse_where(),
|
||||
returning=self._parse_returning(),
|
||||
limit=self._parse_limit(),
|
||||
)
|
||||
|
||||
def _parse_update(self) -> exp.Update:
|
||||
|
|
@ -1790,6 +1798,7 @@ class Parser(metaclass=_Parser):
|
|||
"from": self._parse_from(modifiers=True),
|
||||
"where": self._parse_where(),
|
||||
"returning": self._parse_returning(),
|
||||
"limit": self._parse_limit(),
|
||||
},
|
||||
)
|
||||
|
||||
|
|
@ -2268,6 +2277,33 @@ class Parser(metaclass=_Parser):
|
|||
partition_by=self._parse_partition_by(),
|
||||
)
|
||||
|
||||
def _parse_table_hints(self) -> t.Optional[t.List[exp.Expression]]:
|
||||
hints: t.List[exp.Expression] = []
|
||||
if self._match_pair(TokenType.WITH, TokenType.L_PAREN):
|
||||
# https://learn.microsoft.com/en-us/sql/t-sql/queries/hints-transact-sql-table?view=sql-server-ver16
|
||||
hints.append(
|
||||
self.expression(
|
||||
exp.WithTableHint,
|
||||
expressions=self._parse_csv(
|
||||
lambda: self._parse_function() or self._parse_var(any_token=True)
|
||||
),
|
||||
)
|
||||
)
|
||||
self._match_r_paren()
|
||||
else:
|
||||
# https://dev.mysql.com/doc/refman/8.0/en/index-hints.html
|
||||
while self._match_set(self.TABLE_INDEX_HINT_TOKENS):
|
||||
hint = exp.IndexTableHint(this=self._prev.text.upper())
|
||||
|
||||
self._match_texts({"INDEX", "KEY"})
|
||||
if self._match(TokenType.FOR):
|
||||
hint.set("target", self._advance_any() and self._prev.text.upper())
|
||||
|
||||
hint.set("expressions", self._parse_wrapped_id_vars())
|
||||
hints.append(hint)
|
||||
|
||||
return hints or None
|
||||
|
||||
def _parse_table_part(self, schema: bool = False) -> t.Optional[exp.Expression]:
|
||||
return (
|
||||
(not schema and self._parse_function(optional_parens=False))
|
||||
|
|
@ -2335,12 +2371,7 @@ class Parser(metaclass=_Parser):
|
|||
if not this.args.get("pivots"):
|
||||
this.set("pivots", self._parse_pivots())
|
||||
|
||||
if self._match_pair(TokenType.WITH, TokenType.L_PAREN):
|
||||
this.set(
|
||||
"hints",
|
||||
self._parse_csv(lambda: self._parse_function() or self._parse_var(any_token=True)),
|
||||
)
|
||||
self._match_r_paren()
|
||||
this.set("hints", self._parse_table_hints())
|
||||
|
||||
if not self.ALIAS_POST_TABLESAMPLE:
|
||||
table_sample = self._parse_table_sample()
|
||||
|
|
@ -2610,8 +2641,8 @@ class Parser(metaclass=_Parser):
|
|||
exp.Order, this=this, expressions=self._parse_csv(self._parse_ordered)
|
||||
)
|
||||
|
||||
def _parse_sort(self, exp_class: t.Type[E], *texts: str) -> t.Optional[E]:
|
||||
if not self._match_text_seq(*texts):
|
||||
def _parse_sort(self, exp_class: t.Type[E], token: TokenType) -> t.Optional[E]:
|
||||
if not self._match(token):
|
||||
return None
|
||||
return self.expression(exp_class, expressions=self._parse_csv(self._parse_ordered))
|
||||
|
||||
|
|
@ -3655,7 +3686,11 @@ class Parser(metaclass=_Parser):
|
|||
def _parse_concat(self) -> t.Optional[exp.Expression]:
|
||||
args = self._parse_csv(self._parse_conjunction)
|
||||
if self.CONCAT_NULL_OUTPUTS_STRING:
|
||||
args = [exp.func("COALESCE", arg, exp.Literal.string("")) for arg in args]
|
||||
args = [
|
||||
exp.func("COALESCE", exp.cast(arg, "text"), exp.Literal.string(""))
|
||||
for arg in args
|
||||
if arg
|
||||
]
|
||||
|
||||
# Some dialects (e.g. Trino) don't allow a single-argument CONCAT call, so when
|
||||
# we find such a call we replace it with its argument.
|
||||
|
|
@ -4553,13 +4588,16 @@ class Parser(metaclass=_Parser):
|
|||
curr = self._curr.text.upper()
|
||||
key = curr.split(" ")
|
||||
this.append(curr)
|
||||
|
||||
self._advance()
|
||||
result, trie = in_trie(trie, key)
|
||||
if result == 0:
|
||||
if result == TrieResult.FAILED:
|
||||
break
|
||||
if result == 2:
|
||||
|
||||
if result == TrieResult.EXISTS:
|
||||
subparser = parsers[" ".join(this)]
|
||||
return subparser
|
||||
|
||||
self._retreat(index)
|
||||
return None
|
||||
|
||||
|
|
|
|||
|
|
@ -6,10 +6,10 @@ import typing as t
|
|||
import sqlglot
|
||||
from sqlglot import expressions as exp
|
||||
from sqlglot._typing import T
|
||||
from sqlglot.dialects.dialect import RESOLVES_IDENTIFIERS_AS_UPPERCASE
|
||||
from sqlglot.dialects.dialect import Dialect
|
||||
from sqlglot.errors import ParseError, SchemaError
|
||||
from sqlglot.helper import dict_depth
|
||||
from sqlglot.trie import in_trie, new_trie
|
||||
from sqlglot.trie import TrieResult, in_trie, new_trie
|
||||
|
||||
if t.TYPE_CHECKING:
|
||||
from sqlglot.dataframe.sql.types import StructType
|
||||
|
|
@ -135,10 +135,10 @@ class AbstractMappingSchema(t.Generic[T]):
|
|||
parts = self.table_parts(table)[0 : len(self.supported_table_args)]
|
||||
value, trie = in_trie(self.mapping_trie if trie is None else trie, parts)
|
||||
|
||||
if value == 0:
|
||||
if value == TrieResult.FAILED:
|
||||
return None
|
||||
|
||||
if value == 1:
|
||||
if value == TrieResult.PREFIX:
|
||||
possibilities = flatten_schema(trie, depth=dict_depth(trie) - 1)
|
||||
|
||||
if len(possibilities) == 1:
|
||||
|
|
@ -289,7 +289,7 @@ class MappingSchema(AbstractMappingSchema[t.Dict[str, str]], Schema):
|
|||
|
||||
def _normalize(self, schema: t.Dict) -> t.Dict:
|
||||
"""
|
||||
Converts all identifiers in the schema into lowercase, unless they're quoted.
|
||||
Normalizes all identifiers in the schema.
|
||||
|
||||
Args:
|
||||
schema: the schema to normalize.
|
||||
|
|
@ -304,7 +304,9 @@ class MappingSchema(AbstractMappingSchema[t.Dict[str, str]], Schema):
|
|||
columns = nested_get(schema, *zip(keys, keys))
|
||||
assert columns is not None
|
||||
|
||||
normalized_keys = [self._normalize_name(key, dialect=self.dialect) for key in keys]
|
||||
normalized_keys = [
|
||||
self._normalize_name(key, dialect=self.dialect, is_table=True) for key in keys
|
||||
]
|
||||
for column_name, column_type in columns.items():
|
||||
nested_set(
|
||||
normalized_mapping,
|
||||
|
|
@ -321,12 +323,15 @@ class MappingSchema(AbstractMappingSchema[t.Dict[str, str]], Schema):
|
|||
value = normalized_table.args.get(arg)
|
||||
if isinstance(value, (str, exp.Identifier)):
|
||||
normalized_table.set(
|
||||
arg, exp.to_identifier(self._normalize_name(value, dialect=dialect))
|
||||
arg,
|
||||
exp.to_identifier(self._normalize_name(value, dialect=dialect, is_table=True)),
|
||||
)
|
||||
|
||||
return normalized_table
|
||||
|
||||
def _normalize_name(self, name: str | exp.Identifier, dialect: DialectType = None) -> str:
|
||||
def _normalize_name(
|
||||
self, name: str | exp.Identifier, dialect: DialectType = None, is_table: bool = False
|
||||
) -> str:
|
||||
dialect = dialect or self.dialect
|
||||
|
||||
try:
|
||||
|
|
@ -335,11 +340,12 @@ class MappingSchema(AbstractMappingSchema[t.Dict[str, str]], Schema):
|
|||
return name if isinstance(name, str) else name.name
|
||||
|
||||
name = identifier.name
|
||||
|
||||
if not self.normalize or identifier.quoted:
|
||||
if not self.normalize:
|
||||
return name
|
||||
|
||||
return name.upper() if dialect in RESOLVES_IDENTIFIERS_AS_UPPERCASE else name.lower()
|
||||
# This can be useful for normalize_identifier
|
||||
identifier.meta["is_table"] = is_table
|
||||
return Dialect.get_or_raise(dialect).normalize_identifier(identifier).name
|
||||
|
||||
def _depth(self) -> int:
|
||||
# The columns themselves are a mapping, but we don't want to include those
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ import typing as t
|
|||
|
||||
# The generic time format is based on python time.strftime.
|
||||
# https://docs.python.org/3/library/time.html#time.strftime
|
||||
from sqlglot.trie import in_trie, new_trie
|
||||
from sqlglot.trie import TrieResult, in_trie, new_trie
|
||||
|
||||
|
||||
def format_time(
|
||||
|
|
@ -37,7 +37,7 @@ def format_time(
|
|||
chars = string[start:end]
|
||||
result, current = in_trie(current, chars[-1])
|
||||
|
||||
if result == 0:
|
||||
if result == TrieResult.FAILED:
|
||||
if sym:
|
||||
end -= 1
|
||||
chars = sym
|
||||
|
|
@ -45,11 +45,12 @@ def format_time(
|
|||
start += len(chars)
|
||||
chunks.append(chars)
|
||||
current = trie
|
||||
elif result == 2:
|
||||
elif result == TrieResult.EXISTS:
|
||||
sym = chars
|
||||
|
||||
end += 1
|
||||
|
||||
if result and end > size:
|
||||
if result != TrieResult.FAILED and end > size:
|
||||
chunks.append(chars)
|
||||
|
||||
return "".join(mapping.get(chars, chars) for chars in chunks)
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import typing as t
|
|||
from enum import auto
|
||||
|
||||
from sqlglot.helper import AutoName
|
||||
from sqlglot.trie import in_trie, new_trie
|
||||
from sqlglot.trie import TrieResult, in_trie, new_trie
|
||||
|
||||
|
||||
class TokenType(AutoName):
|
||||
|
|
@ -137,6 +137,7 @@ class TokenType(AutoName):
|
|||
BIGSERIAL = auto()
|
||||
XML = auto()
|
||||
UNIQUEIDENTIFIER = auto()
|
||||
USERDEFINED = auto()
|
||||
MONEY = auto()
|
||||
SMALLMONEY = auto()
|
||||
ROWVERSION = auto()
|
||||
|
|
@ -163,6 +164,7 @@ class TokenType(AutoName):
|
|||
CACHE = auto()
|
||||
CASE = auto()
|
||||
CHARACTER_SET = auto()
|
||||
CLUSTER_BY = auto()
|
||||
COLLATE = auto()
|
||||
COMMAND = auto()
|
||||
COMMENT = auto()
|
||||
|
|
@ -182,6 +184,7 @@ class TokenType(AutoName):
|
|||
DESCRIBE = auto()
|
||||
DICTIONARY = auto()
|
||||
DISTINCT = auto()
|
||||
DISTRIBUTE_BY = auto()
|
||||
DIV = auto()
|
||||
DROP = auto()
|
||||
ELSE = auto()
|
||||
|
|
@ -196,6 +199,7 @@ class TokenType(AutoName):
|
|||
FINAL = auto()
|
||||
FIRST = auto()
|
||||
FOR = auto()
|
||||
FORCE = auto()
|
||||
FOREIGN_KEY = auto()
|
||||
FORMAT = auto()
|
||||
FROM = auto()
|
||||
|
|
@ -208,6 +212,7 @@ class TokenType(AutoName):
|
|||
HAVING = auto()
|
||||
HINT = auto()
|
||||
IF = auto()
|
||||
IGNORE = auto()
|
||||
ILIKE = auto()
|
||||
ILIKE_ANY = auto()
|
||||
IN = auto()
|
||||
|
|
@ -282,6 +287,7 @@ class TokenType(AutoName):
|
|||
SHOW = auto()
|
||||
SIMILAR_TO = auto()
|
||||
SOME = auto()
|
||||
SORT_BY = auto()
|
||||
STRUCT = auto()
|
||||
TABLE_SAMPLE = auto()
|
||||
TEMPORARY = auto()
|
||||
|
|
@ -509,6 +515,7 @@ class Tokenizer(metaclass=_Tokenizer):
|
|||
"UNCACHE": TokenType.UNCACHE,
|
||||
"CASE": TokenType.CASE,
|
||||
"CHARACTER SET": TokenType.CHARACTER_SET,
|
||||
"CLUSTER BY": TokenType.CLUSTER_BY,
|
||||
"COLLATE": TokenType.COLLATE,
|
||||
"COLUMN": TokenType.COLUMN,
|
||||
"COMMIT": TokenType.COMMIT,
|
||||
|
|
@ -526,6 +533,7 @@ class Tokenizer(metaclass=_Tokenizer):
|
|||
"DESC": TokenType.DESC,
|
||||
"DESCRIBE": TokenType.DESCRIBE,
|
||||
"DISTINCT": TokenType.DISTINCT,
|
||||
"DISTRIBUTE BY": TokenType.DISTRIBUTE_BY,
|
||||
"DIV": TokenType.DIV,
|
||||
"DROP": TokenType.DROP,
|
||||
"ELSE": TokenType.ELSE,
|
||||
|
|
@ -617,6 +625,7 @@ class Tokenizer(metaclass=_Tokenizer):
|
|||
"SHOW": TokenType.SHOW,
|
||||
"SIMILAR TO": TokenType.SIMILAR_TO,
|
||||
"SOME": TokenType.SOME,
|
||||
"SORT BY": TokenType.SORT_BY,
|
||||
"TABLE": TokenType.TABLE,
|
||||
"TABLESAMPLE": TokenType.TABLE_SAMPLE,
|
||||
"TEMP": TokenType.TEMPORARY,
|
||||
|
|
@ -717,6 +726,7 @@ class Tokenizer(metaclass=_Tokenizer):
|
|||
"PREPARE": TokenType.COMMAND,
|
||||
"TRUNCATE": TokenType.COMMAND,
|
||||
"VACUUM": TokenType.COMMAND,
|
||||
"USER-DEFINED": TokenType.USERDEFINED,
|
||||
}
|
||||
|
||||
WHITE_SPACE: t.Dict[t.Optional[str], TokenType] = {
|
||||
|
|
@ -905,13 +915,13 @@ class Tokenizer(metaclass=_Tokenizer):
|
|||
|
||||
while chars:
|
||||
if skip:
|
||||
result = 1
|
||||
result = TrieResult.PREFIX
|
||||
else:
|
||||
result, trie = in_trie(trie, char.upper())
|
||||
|
||||
if result == 0:
|
||||
if result == TrieResult.FAILED:
|
||||
break
|
||||
if result == 2:
|
||||
if result == TrieResult.EXISTS:
|
||||
word = chars
|
||||
|
||||
size += 1
|
||||
|
|
|
|||
|
|
@ -1,14 +1,21 @@
|
|||
import typing as t
|
||||
from enum import Enum, auto
|
||||
|
||||
key = t.Sequence[t.Hashable]
|
||||
|
||||
|
||||
class TrieResult(Enum):
|
||||
FAILED = auto()
|
||||
PREFIX = auto()
|
||||
EXISTS = auto()
|
||||
|
||||
|
||||
def new_trie(keywords: t.Iterable[key], trie: t.Optional[t.Dict] = None) -> t.Dict:
|
||||
"""
|
||||
Creates a new trie out of a collection of keywords.
|
||||
|
||||
The trie is represented as a sequence of nested dictionaries keyed by either single character
|
||||
strings, or by 0, which is used to designate that a keyword is in the trie.
|
||||
The trie is represented as a sequence of nested dictionaries keyed by either single
|
||||
character strings, or by 0, which is used to designate that a keyword is in the trie.
|
||||
|
||||
Example:
|
||||
>>> new_trie(["bla", "foo", "blab"])
|
||||
|
|
@ -25,46 +32,50 @@ def new_trie(keywords: t.Iterable[key], trie: t.Optional[t.Dict] = None) -> t.Di
|
|||
|
||||
for key in keywords:
|
||||
current = trie
|
||||
|
||||
for char in key:
|
||||
current = current.setdefault(char, {})
|
||||
|
||||
current[0] = True
|
||||
|
||||
return trie
|
||||
|
||||
|
||||
def in_trie(trie: t.Dict, key: key) -> t.Tuple[int, t.Dict]:
|
||||
def in_trie(trie: t.Dict, key: key) -> t.Tuple[TrieResult, t.Dict]:
|
||||
"""
|
||||
Checks whether a key is in a trie.
|
||||
|
||||
Examples:
|
||||
>>> in_trie(new_trie(["cat"]), "bob")
|
||||
(0, {'c': {'a': {'t': {0: True}}}})
|
||||
(<TrieResult.FAILED: 1>, {'c': {'a': {'t': {0: True}}}})
|
||||
|
||||
>>> in_trie(new_trie(["cat"]), "ca")
|
||||
(1, {'t': {0: True}})
|
||||
(<TrieResult.PREFIX: 2>, {'t': {0: True}})
|
||||
|
||||
>>> in_trie(new_trie(["cat"]), "cat")
|
||||
(2, {0: True})
|
||||
(<TrieResult.EXISTS: 3>, {0: True})
|
||||
|
||||
Args:
|
||||
trie: the trie to be searched.
|
||||
key: the target key.
|
||||
trie: The trie to be searched.
|
||||
key: The target key.
|
||||
|
||||
Returns:
|
||||
A pair `(value, subtrie)`, where `subtrie` is the sub-trie we get at the point where the search stops, and `value`
|
||||
is either 0 (search was unsuccessful), 1 (`value` is a prefix of a keyword in `trie`) or 2 (`key is in `trie`).
|
||||
A pair `(value, subtrie)`, where `subtrie` is the sub-trie we get at the point
|
||||
where the search stops, and `value` is a TrieResult value that can be one of:
|
||||
|
||||
- TrieResult.FAILED: the search was unsuccessful
|
||||
- TrieResult.PREFIX: `value` is a prefix of a keyword in `trie`
|
||||
- TrieResult.EXISTS: `key` exists in `trie`
|
||||
"""
|
||||
if not key:
|
||||
return (0, trie)
|
||||
return (TrieResult.FAILED, trie)
|
||||
|
||||
current = trie
|
||||
|
||||
for char in key:
|
||||
if char not in current:
|
||||
return (0, current)
|
||||
return (TrieResult.FAILED, current)
|
||||
current = current[char]
|
||||
|
||||
if 0 in current:
|
||||
return (2, current)
|
||||
return (1, current)
|
||||
return (TrieResult.EXISTS, current)
|
||||
|
||||
return (TrieResult.PREFIX, current)
|
||||
|
|
|
|||
|
|
@ -135,9 +135,9 @@ class DataFrameValidator(unittest.TestCase):
|
|||
data=district_data, schema=cls.sqlglot_district_schema
|
||||
)
|
||||
cls.df_district.createOrReplaceTempView("district")
|
||||
sqlglot.schema.add_table("employee", cls.sqlglot_employee_schema)
|
||||
sqlglot.schema.add_table("store", cls.sqlglot_store_schema)
|
||||
sqlglot.schema.add_table("district", cls.sqlglot_district_schema)
|
||||
sqlglot.schema.add_table("employee", cls.sqlglot_employee_schema, dialect="spark")
|
||||
sqlglot.schema.add_table("store", cls.sqlglot_store_schema, dialect="spark")
|
||||
sqlglot.schema.add_table("district", cls.sqlglot_district_schema, dialect="spark")
|
||||
|
||||
def setUp(self) -> None:
|
||||
warnings.filterwarnings("ignore", category=ResourceWarning)
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ class TestDataFrameWriter(DataFrameSQLValidator):
|
|||
|
||||
@mock.patch("sqlglot.schema", MappingSchema())
|
||||
def test_insertInto_byName(self):
|
||||
sqlglot.schema.add_table("table_name", {"employee_id": "INT"})
|
||||
sqlglot.schema.add_table("table_name", {"employee_id": "INT"}, dialect="spark")
|
||||
df = self.df_employee.write.byName.insertInto("table_name")
|
||||
expected = "INSERT INTO table_name SELECT `a1`.`employee_id` AS `employee_id` FROM VALUES (1, 'Jack', 'Shephard', 37, 1), (2, 'John', 'Locke', 65, 1), (3, 'Kate', 'Austen', 37, 2), (4, 'Claire', 'Littleton', 27, 2), (5, 'Hugo', 'Reyes', 29, 100) AS `a1`(`employee_id`, `fname`, `lname`, `age`, `store_id`)"
|
||||
self.compare_sql(df, expected)
|
||||
|
|
@ -88,8 +88,8 @@ class TestDataFrameWriter(DataFrameSQLValidator):
|
|||
self.compare_sql(df, expected_statements)
|
||||
|
||||
def test_quotes(self):
|
||||
sqlglot.schema.add_table('"Test"', {'"ID"': "STRING"})
|
||||
df = self.spark.table('"Test"')
|
||||
sqlglot.schema.add_table("`Test`", {"`ID`": "STRING"}, dialect="spark")
|
||||
df = self.spark.table("`Test`")
|
||||
self.compare_sql(
|
||||
df.select(df['"ID"']), ["SELECT `Test`.`ID` AS `ID` FROM `Test` AS `Test`"]
|
||||
df.select(df["`ID`"]), ["SELECT `test`.`id` AS `id` FROM `test` AS `test`"]
|
||||
)
|
||||
|
|
|
|||
|
|
@ -71,7 +71,7 @@ class TestDataframeSession(DataFrameSQLValidator):
|
|||
@mock.patch("sqlglot.schema", MappingSchema())
|
||||
def test_sql_select_only(self):
|
||||
query = "SELECT cola, colb FROM table"
|
||||
sqlglot.schema.add_table("table", {"cola": "string", "colb": "string"})
|
||||
sqlglot.schema.add_table("table", {"cola": "string", "colb": "string"}, dialect="spark")
|
||||
df = self.spark.sql(query)
|
||||
self.assertEqual(
|
||||
"SELECT `table`.`cola` AS `cola`, `table`.`colb` AS `colb` FROM `table` AS `table`",
|
||||
|
|
@ -80,17 +80,17 @@ class TestDataframeSession(DataFrameSQLValidator):
|
|||
|
||||
@mock.patch("sqlglot.schema", MappingSchema())
|
||||
def test_select_quoted(self):
|
||||
sqlglot.schema.add_table('"TEST"', {"name": "string"})
|
||||
sqlglot.schema.add_table("`TEST`", {"name": "string"}, dialect="spark")
|
||||
|
||||
self.assertEqual(
|
||||
SparkSession().table('"TEST"').select(F.col("name")).sql(dialect="snowflake")[0],
|
||||
'''SELECT "TEST"."name" AS "name" FROM "TEST" AS "TEST"''',
|
||||
SparkSession().table("`TEST`").select(F.col("name")).sql(dialect="snowflake")[0],
|
||||
'''SELECT "test"."name" AS "name" FROM "test" AS "test"''',
|
||||
)
|
||||
|
||||
@mock.patch("sqlglot.schema", MappingSchema())
|
||||
def test_sql_with_aggs(self):
|
||||
query = "SELECT cola, colb FROM table"
|
||||
sqlglot.schema.add_table("table", {"cola": "string", "colb": "string"})
|
||||
sqlglot.schema.add_table("table", {"cola": "string", "colb": "string"}, dialect="spark")
|
||||
df = self.spark.sql(query).groupBy(F.col("cola")).agg(F.sum("colb"))
|
||||
self.assertEqual(
|
||||
"WITH t38189 AS (SELECT cola, colb FROM table), t42330 AS (SELECT cola, colb FROM t38189) SELECT cola, SUM(colb) FROM t42330 GROUP BY cola",
|
||||
|
|
@ -100,7 +100,7 @@ class TestDataframeSession(DataFrameSQLValidator):
|
|||
@mock.patch("sqlglot.schema", MappingSchema())
|
||||
def test_sql_create(self):
|
||||
query = "CREATE TABLE new_table AS WITH t1 AS (SELECT cola, colb FROM table) SELECT cola, colb, FROM t1"
|
||||
sqlglot.schema.add_table("table", {"cola": "string", "colb": "string"})
|
||||
sqlglot.schema.add_table("table", {"cola": "string", "colb": "string"}, dialect="spark")
|
||||
df = self.spark.sql(query)
|
||||
expected = "CREATE TABLE new_table AS SELECT `table`.`cola` AS `cola`, `table`.`colb` AS `colb` FROM `table` AS `table`"
|
||||
self.compare_sql(df, expected)
|
||||
|
|
@ -108,7 +108,7 @@ class TestDataframeSession(DataFrameSQLValidator):
|
|||
@mock.patch("sqlglot.schema", MappingSchema())
|
||||
def test_sql_insert(self):
|
||||
query = "WITH t1 AS (SELECT cola, colb FROM table) INSERT INTO new_table SELECT cola, colb FROM t1"
|
||||
sqlglot.schema.add_table("table", {"cola": "string", "colb": "string"})
|
||||
sqlglot.schema.add_table("table", {"cola": "string", "colb": "string"}, dialect="spark")
|
||||
df = self.spark.sql(query)
|
||||
expected = "INSERT INTO new_table SELECT `table`.`cola` AS `cola`, `table`.`colb` AS `colb` FROM `table` AS `table`"
|
||||
self.compare_sql(df, expected)
|
||||
|
|
|
|||
|
|
@ -23,6 +23,14 @@ class TestBigQuery(Validator):
|
|||
self.validate_identity("SELECT b'abc'")
|
||||
self.validate_identity("""SELECT * FROM UNNEST(ARRAY<STRUCT<x INT64>>[1, 2])""")
|
||||
self.validate_identity("SELECT AS STRUCT 1 AS a, 2 AS b")
|
||||
self.validate_all(
|
||||
"SELECT AS STRUCT ARRAY(SELECT AS STRUCT b FROM x) AS y FROM z",
|
||||
write={
|
||||
"": "SELECT AS STRUCT ARRAY(SELECT AS STRUCT b FROM x) AS y FROM z",
|
||||
"bigquery": "SELECT AS STRUCT ARRAY(SELECT AS STRUCT b FROM x) AS y FROM z",
|
||||
"duckdb": "SELECT {'y': ARRAY(SELECT {'b': b} FROM x)} FROM z",
|
||||
},
|
||||
)
|
||||
self.validate_identity("SELECT DISTINCT AS STRUCT 1 AS a, 2 AS b")
|
||||
self.validate_identity("SELECT AS VALUE STRUCT(1 AS a, 2 AS b)")
|
||||
self.validate_identity("SELECT STRUCT<ARRAY<STRING>>(['2023-01-17'])")
|
||||
|
|
@ -116,6 +124,21 @@ class TestBigQuery(Validator):
|
|||
with self.assertRaises(ValueError):
|
||||
transpile("'\\'", read="bigquery")
|
||||
|
||||
self.validate_all(
|
||||
"r'x\\''",
|
||||
write={
|
||||
"bigquery": "r'x\\''",
|
||||
"hive": "'x\\''",
|
||||
},
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"r'x\\y'",
|
||||
write={
|
||||
"bigquery": "r'x\\y'",
|
||||
"hive": "'x\\\\y'",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"'\\\\'",
|
||||
write={
|
||||
|
|
@ -458,6 +481,21 @@ class TestBigQuery(Validator):
|
|||
"SELECT * FROM UNNEST([1]) WITH OFFSET y",
|
||||
write={"bigquery": "SELECT * FROM UNNEST([1]) WITH OFFSET AS y"},
|
||||
)
|
||||
self.validate_all(
|
||||
"GENERATE_ARRAY(1, 4)",
|
||||
read={"bigquery": "GENERATE_ARRAY(1, 4)"},
|
||||
write={"duckdb": "GENERATE_SERIES(1, 4)"},
|
||||
)
|
||||
self.validate_all(
|
||||
"TO_JSON_STRING(x)",
|
||||
read={"bigquery": "TO_JSON_STRING(x)"},
|
||||
write={
|
||||
"bigquery": "TO_JSON_STRING(x)",
|
||||
"duckdb": "CAST(TO_JSON(x) AS TEXT)",
|
||||
"presto": "JSON_FORMAT(x)",
|
||||
"spark": "TO_JSON(x)",
|
||||
},
|
||||
)
|
||||
|
||||
def test_user_defined_functions(self):
|
||||
self.validate_identity(
|
||||
|
|
|
|||
|
|
@ -53,7 +53,7 @@ class TestClickhouse(Validator):
|
|||
)
|
||||
|
||||
self.validate_all(
|
||||
"CONCAT(CASE WHEN COALESCE(a, '') IS NULL THEN COALESCE(a, '') ELSE CAST(COALESCE(a, '') AS TEXT) END, CASE WHEN COALESCE(b, '') IS NULL THEN COALESCE(b, '') ELSE CAST(COALESCE(b, '') AS TEXT) END)",
|
||||
"CONCAT(CASE WHEN COALESCE(CAST(a AS TEXT), '') IS NULL THEN COALESCE(CAST(a AS TEXT), '') ELSE CAST(COALESCE(CAST(a AS TEXT), '') AS TEXT) END, CASE WHEN COALESCE(CAST(b AS TEXT), '') IS NULL THEN COALESCE(CAST(b AS TEXT), '') ELSE CAST(COALESCE(CAST(b AS TEXT), '') AS TEXT) END)",
|
||||
read={"postgres": "CONCAT(a, b)"},
|
||||
)
|
||||
self.validate_all(
|
||||
|
|
|
|||
|
|
@ -485,7 +485,7 @@ class TestDialect(Validator):
|
|||
"duckdb": "CAST(x AS DATE)",
|
||||
"hive": "TO_DATE(x)",
|
||||
"postgres": "CAST(x AS DATE)",
|
||||
"presto": "CAST(SUBSTR(CAST(x AS VARCHAR), 1, 10) AS DATE)",
|
||||
"presto": "CAST(CAST(x AS TIMESTAMP) AS DATE)",
|
||||
"snowflake": "CAST(x AS DATE)",
|
||||
},
|
||||
)
|
||||
|
|
@ -749,14 +749,14 @@ class TestDialect(Validator):
|
|||
"drill": "DATE_ADD(CAST('2021-02-01' AS DATE), INTERVAL 1 DAY)",
|
||||
"duckdb": "CAST('2021-02-01' AS DATE) + INTERVAL 1 DAY",
|
||||
"hive": "DATE_ADD('2021-02-01', 1)",
|
||||
"presto": "DATE_ADD('DAY', 1, DATE_PARSE(SUBSTR('2021-02-01', 1, 10), '%Y-%m-%d'))",
|
||||
"presto": "DATE_ADD('DAY', 1, CAST(CAST('2021-02-01' AS TIMESTAMP) AS DATE))",
|
||||
"spark": "DATE_ADD('2021-02-01', 1)",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"TS_OR_DS_ADD(x, 1, 'DAY')",
|
||||
write={
|
||||
"presto": "DATE_ADD('DAY', 1, DATE_PARSE(SUBSTR(CAST(x AS VARCHAR), 1, 10), '%Y-%m-%d'))",
|
||||
"presto": "DATE_ADD('DAY', 1, CAST(CAST(x AS TIMESTAMP) AS DATE))",
|
||||
"hive": "DATE_ADD(x, 1)",
|
||||
},
|
||||
)
|
||||
|
|
@ -1192,7 +1192,7 @@ class TestDialect(Validator):
|
|||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"COALESCE(a, '')",
|
||||
"COALESCE(CAST(a AS TEXT), '')",
|
||||
read={
|
||||
"drill": "CONCAT(a)",
|
||||
"duckdb": "CONCAT(a)",
|
||||
|
|
@ -1300,7 +1300,9 @@ class TestDialect(Validator):
|
|||
self.validate_all(
|
||||
"SELECT x FROM y LIMIT 10",
|
||||
read={
|
||||
"teradata": "SELECT TOP 10 x FROM y",
|
||||
"tsql": "SELECT TOP 10 x FROM y",
|
||||
"snowflake": "SELECT TOP 10 x FROM y",
|
||||
},
|
||||
write={
|
||||
"sqlite": "SELECT x FROM y LIMIT 10",
|
||||
|
|
|
|||
|
|
@ -444,6 +444,7 @@ class TestDuckDB(Validator):
|
|||
|
||||
def test_array(self):
|
||||
self.validate_identity("ARRAY(SELECT id FROM t)")
|
||||
self.validate_identity("ARRAY((SELECT id FROM t))")
|
||||
|
||||
def test_cast(self):
|
||||
self.validate_identity("CAST(x AS REAL)")
|
||||
|
|
|
|||
|
|
@ -4,17 +4,6 @@ from tests.dialects.test_dialect import Validator
|
|||
class TestHive(Validator):
|
||||
dialect = "hive"
|
||||
|
||||
def test_hive(self):
|
||||
self.validate_identity("SELECT * FROM test DISTRIBUTE BY y SORT BY x DESC ORDER BY l")
|
||||
self.validate_identity(
|
||||
"SELECT * FROM test WHERE RAND() <= 0.1 DISTRIBUTE BY RAND() SORT BY RAND()"
|
||||
)
|
||||
self.validate_identity("(SELECT 1 UNION SELECT 2) DISTRIBUTE BY z")
|
||||
self.validate_identity("(SELECT 1 UNION SELECT 2) DISTRIBUTE BY z SORT BY x")
|
||||
self.validate_identity("(SELECT 1 UNION SELECT 2) CLUSTER BY y DESC")
|
||||
self.validate_identity("SELECT * FROM test CLUSTER BY y")
|
||||
self.validate_identity("(SELECT 1 UNION SELECT 2) SORT BY z")
|
||||
|
||||
def test_bits(self):
|
||||
self.validate_all(
|
||||
"x & 1",
|
||||
|
|
@ -288,7 +277,7 @@ class TestHive(Validator):
|
|||
"DATEDIFF(a, b)",
|
||||
write={
|
||||
"duckdb": "DATE_DIFF('day', CAST(b AS DATE), CAST(a AS DATE))",
|
||||
"presto": "DATE_DIFF('day', CAST(SUBSTR(CAST(b AS VARCHAR), 1, 10) AS DATE), CAST(SUBSTR(CAST(a AS VARCHAR), 1, 10) AS DATE))",
|
||||
"presto": "DATE_DIFF('day', CAST(CAST(b AS TIMESTAMP) AS DATE), CAST(CAST(a AS TIMESTAMP) AS DATE))",
|
||||
"hive": "DATEDIFF(TO_DATE(a), TO_DATE(b))",
|
||||
"spark": "DATEDIFF(TO_DATE(a), TO_DATE(b))",
|
||||
"": "DATEDIFF(TS_OR_DS_TO_DATE(a), TS_OR_DS_TO_DATE(b))",
|
||||
|
|
@ -316,7 +305,7 @@ class TestHive(Validator):
|
|||
"DATE_ADD('2020-01-01', 1)",
|
||||
write={
|
||||
"duckdb": "CAST('2020-01-01' AS DATE) + INTERVAL 1 DAY",
|
||||
"presto": "DATE_ADD('DAY', 1, DATE_PARSE(SUBSTR('2020-01-01', 1, 10), '%Y-%m-%d'))",
|
||||
"presto": "DATE_ADD('DAY', 1, CAST(CAST('2020-01-01' AS TIMESTAMP) AS DATE))",
|
||||
"hive": "DATE_ADD('2020-01-01', 1)",
|
||||
"spark": "DATE_ADD('2020-01-01', 1)",
|
||||
"": "TS_OR_DS_ADD('2020-01-01', 1, 'DAY')",
|
||||
|
|
@ -326,7 +315,7 @@ class TestHive(Validator):
|
|||
"DATE_SUB('2020-01-01', 1)",
|
||||
write={
|
||||
"duckdb": "CAST('2020-01-01' AS DATE) + INTERVAL (1 * -1) DAY",
|
||||
"presto": "DATE_ADD('DAY', 1 * -1, DATE_PARSE(SUBSTR('2020-01-01', 1, 10), '%Y-%m-%d'))",
|
||||
"presto": "DATE_ADD('DAY', 1 * -1, CAST(CAST('2020-01-01' AS TIMESTAMP) AS DATE))",
|
||||
"hive": "DATE_ADD('2020-01-01', 1 * -1)",
|
||||
"spark": "DATE_ADD('2020-01-01', 1 * -1)",
|
||||
"": "TS_OR_DS_ADD('2020-01-01', 1 * -1, 'DAY')",
|
||||
|
|
@ -341,7 +330,7 @@ class TestHive(Validator):
|
|||
"DATEDIFF(TO_DATE(y), x)",
|
||||
write={
|
||||
"duckdb": "DATE_DIFF('day', CAST(x AS DATE), CAST(CAST(y AS DATE) AS DATE))",
|
||||
"presto": "DATE_DIFF('day', CAST(SUBSTR(CAST(x AS VARCHAR), 1, 10) AS DATE), CAST(SUBSTR(CAST(CAST(SUBSTR(CAST(y AS VARCHAR), 1, 10) AS DATE) AS VARCHAR), 1, 10) AS DATE))",
|
||||
"presto": "DATE_DIFF('day', CAST(CAST(x AS TIMESTAMP) AS DATE), CAST(CAST(CAST(CAST(y AS TIMESTAMP) AS DATE) AS TIMESTAMP) AS DATE))",
|
||||
"hive": "DATEDIFF(TO_DATE(TO_DATE(y)), TO_DATE(x))",
|
||||
"spark": "DATEDIFF(TO_DATE(TO_DATE(y)), TO_DATE(x))",
|
||||
"": "DATEDIFF(TS_OR_DS_TO_DATE(TS_OR_DS_TO_DATE(y)), TS_OR_DS_TO_DATE(x))",
|
||||
|
|
@ -363,7 +352,7 @@ class TestHive(Validator):
|
|||
f"{unit}(x)",
|
||||
write={
|
||||
"duckdb": f"{unit}(CAST(x AS DATE))",
|
||||
"presto": f"{unit}(CAST(SUBSTR(CAST(x AS VARCHAR), 1, 10) AS DATE))",
|
||||
"presto": f"{unit}(CAST(CAST(x AS TIMESTAMP) AS DATE))",
|
||||
"hive": f"{unit}(TO_DATE(x))",
|
||||
"spark": f"{unit}(TO_DATE(x))",
|
||||
},
|
||||
|
|
@ -381,6 +370,16 @@ class TestHive(Validator):
|
|||
)
|
||||
|
||||
def test_hive(self):
|
||||
self.validate_identity("SELECT * FROM test DISTRIBUTE BY y SORT BY x DESC ORDER BY l")
|
||||
self.validate_identity(
|
||||
"SELECT * FROM test WHERE RAND() <= 0.1 DISTRIBUTE BY RAND() SORT BY RAND()"
|
||||
)
|
||||
self.validate_identity("(SELECT 1 UNION SELECT 2) DISTRIBUTE BY z")
|
||||
self.validate_identity("(SELECT 1 UNION SELECT 2) DISTRIBUTE BY z SORT BY x")
|
||||
self.validate_identity("(SELECT 1 UNION SELECT 2) CLUSTER BY y DESC")
|
||||
self.validate_identity("SELECT * FROM test CLUSTER BY y")
|
||||
|
||||
self.validate_identity("(SELECT 1 UNION SELECT 2) SORT BY z")
|
||||
self.validate_identity(
|
||||
"INSERT OVERWRITE TABLE zipcodes PARTITION(state = '0') VALUES (896, 'US', 'TAMPA', 33607)"
|
||||
)
|
||||
|
|
|
|||
|
|
@ -6,6 +6,8 @@ class TestMySQL(Validator):
|
|||
dialect = "mysql"
|
||||
|
||||
def test_ddl(self):
|
||||
self.validate_identity("UPDATE items SET items.price = 0 WHERE items.id >= 5 LIMIT 10")
|
||||
self.validate_identity("DELETE FROM t WHERE a <= 10 LIMIT 10")
|
||||
self.validate_identity(
|
||||
"INSERT INTO x VALUES (1, 'a', 2.0) ON DUPLICATE KEY UPDATE SET x.id = 1"
|
||||
)
|
||||
|
|
@ -61,6 +63,22 @@ class TestMySQL(Validator):
|
|||
"SELECT * FROM t1, t2, t3 FOR SHARE OF t1 NOWAIT FOR UPDATE OF t2, t3 SKIP LOCKED"
|
||||
)
|
||||
|
||||
# Index hints
|
||||
self.validate_identity(
|
||||
"SELECT * FROM table1 USE INDEX (col1_index, col2_index) WHERE col1 = 1 AND col2 = 2 AND col3 = 3"
|
||||
)
|
||||
self.validate_identity(
|
||||
"SELECT * FROM table1 IGNORE INDEX (col3_index) WHERE col1 = 1 AND col2 = 2 AND col3 = 3"
|
||||
)
|
||||
self.validate_identity(
|
||||
"SELECT * FROM t1 USE INDEX (i1) IGNORE INDEX FOR ORDER BY (i2) ORDER BY a"
|
||||
)
|
||||
self.validate_identity("SELECT * FROM t1 USE INDEX (i1) USE INDEX (i1, i1)")
|
||||
self.validate_identity("SELECT * FROM t1 USE INDEX FOR JOIN (i1) FORCE INDEX FOR JOIN (i2)")
|
||||
self.validate_identity(
|
||||
"SELECT * FROM t1 USE INDEX () IGNORE INDEX (i2) USE INDEX (i1) USE INDEX (i2)"
|
||||
)
|
||||
|
||||
# SET Commands
|
||||
self.validate_identity("SET @var_name = expr")
|
||||
self.validate_identity("SET @name = 43")
|
||||
|
|
@ -80,12 +98,6 @@ class TestMySQL(Validator):
|
|||
self.validate_identity("SET @@SESSION.max_join_size = DEFAULT")
|
||||
self.validate_identity("SET @@SESSION.max_join_size = @@GLOBAL.max_join_size")
|
||||
self.validate_identity("SET @x = 1, SESSION sql_mode = ''")
|
||||
self.validate_identity(
|
||||
"SET GLOBAL sort_buffer_size = 1000000, SESSION sort_buffer_size = 1000000"
|
||||
)
|
||||
self.validate_identity(
|
||||
"SET @@GLOBAL.sort_buffer_size = 1000000, @@LOCAL.sort_buffer_size = 1000000"
|
||||
)
|
||||
self.validate_identity("SET GLOBAL max_connections = 1000, sort_buffer_size = 1000000")
|
||||
self.validate_identity("SET @@GLOBAL.sort_buffer_size = 50000, sort_buffer_size = 1000000")
|
||||
self.validate_identity("SET CHARACTER SET 'utf8'")
|
||||
|
|
@ -101,6 +113,12 @@ class TestMySQL(Validator):
|
|||
self.validate_identity("SET GLOBAL TRANSACTION ISOLATION LEVEL REPEATABLE READ, READ WRITE")
|
||||
self.validate_identity("SELECT SCHEMA()")
|
||||
self.validate_identity("SELECT DATABASE()")
|
||||
self.validate_identity(
|
||||
"SET GLOBAL sort_buffer_size = 1000000, SESSION sort_buffer_size = 1000000"
|
||||
)
|
||||
self.validate_identity(
|
||||
"SET @@GLOBAL.sort_buffer_size = 1000000, @@LOCAL.sort_buffer_size = 1000000"
|
||||
)
|
||||
|
||||
def test_types(self):
|
||||
self.validate_all(
|
||||
|
|
|
|||
|
|
@ -577,13 +577,19 @@ class TestPostgres(Validator):
|
|||
)
|
||||
|
||||
def test_string_concat(self):
|
||||
self.validate_all(
|
||||
"SELECT CONCAT('abcde', 2, NULL, 22)",
|
||||
write={
|
||||
"postgres": "SELECT CONCAT(COALESCE(CAST('abcde' AS TEXT), ''), COALESCE(CAST(2 AS TEXT), ''), COALESCE(CAST(NULL AS TEXT), ''), COALESCE(CAST(22 AS TEXT), ''))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"CONCAT(a, b)",
|
||||
write={
|
||||
"": "CONCAT(COALESCE(a, ''), COALESCE(b, ''))",
|
||||
"duckdb": "CONCAT(COALESCE(a, ''), COALESCE(b, ''))",
|
||||
"postgres": "CONCAT(COALESCE(a, ''), COALESCE(b, ''))",
|
||||
"presto": "CONCAT(CAST(COALESCE(a, '') AS VARCHAR), CAST(COALESCE(b, '') AS VARCHAR))",
|
||||
"": "CONCAT(COALESCE(CAST(a AS TEXT), ''), COALESCE(CAST(b AS TEXT), ''))",
|
||||
"duckdb": "CONCAT(COALESCE(CAST(a AS TEXT), ''), COALESCE(CAST(b AS TEXT), ''))",
|
||||
"postgres": "CONCAT(COALESCE(CAST(a AS TEXT), ''), COALESCE(CAST(b AS TEXT), ''))",
|
||||
"presto": "CONCAT(CAST(COALESCE(CAST(a AS VARCHAR), '') AS VARCHAR), CAST(COALESCE(CAST(b AS VARCHAR), '') AS VARCHAR))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
|
|
|
|||
|
|
@ -7,11 +7,11 @@ class TestPresto(Validator):
|
|||
|
||||
def test_cast(self):
|
||||
self.validate_all(
|
||||
"SELECT DATE_DIFF('week', CAST(SUBSTR(CAST('2009-01-01' AS VARCHAR), 1, 10) AS DATE), CAST(SUBSTR(CAST('2009-12-31' AS VARCHAR), 1, 10) AS DATE))",
|
||||
"SELECT DATE_DIFF('week', CAST(CAST('2009-01-01' AS TIMESTAMP) AS DATE), CAST(CAST('2009-12-31' AS TIMESTAMP) AS DATE))",
|
||||
read={"redshift": "SELECT DATEDIFF(week, '2009-01-01', '2009-12-31')"},
|
||||
)
|
||||
self.validate_all(
|
||||
"SELECT DATE_ADD('month', 18, CAST(SUBSTR(CAST('2008-02-28' AS VARCHAR), 1, 10) AS DATE))",
|
||||
"SELECT DATE_ADD('month', 18, CAST(CAST('2008-02-28' AS TIMESTAMP) AS DATE))",
|
||||
read={"redshift": "SELECT DATEADD(month, 18, '2008-02-28')"},
|
||||
)
|
||||
self.validate_all(
|
||||
|
|
@ -664,16 +664,31 @@ class TestPresto(Validator):
|
|||
"spark": "TO_JSON(x)",
|
||||
},
|
||||
write={
|
||||
"bigquery": "TO_JSON_STRING(x)",
|
||||
"duckdb": "CAST(TO_JSON(x) AS TEXT)",
|
||||
"presto": "JSON_FORMAT(x)",
|
||||
"spark": "TO_JSON(x)",
|
||||
},
|
||||
)
|
||||
|
||||
self.validate_all(
|
||||
"JSON_FORMAT(JSON 'x')",
|
||||
"""JSON_FORMAT(JSON '"x"')""",
|
||||
write={
|
||||
"presto": "JSON_FORMAT(CAST('x' AS JSON))",
|
||||
"spark": "TO_JSON('x')",
|
||||
"bigquery": """TO_JSON_STRING(CAST('"x"' AS JSON))""",
|
||||
"duckdb": """CAST(TO_JSON(CAST('"x"' AS JSON)) AS TEXT)""",
|
||||
"presto": """JSON_FORMAT(CAST('"x"' AS JSON))""",
|
||||
"spark": """REGEXP_EXTRACT(TO_JSON(FROM_JSON('["x"]', SCHEMA_OF_JSON('["x"]'))), '^.(.*).$', 1)""",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"""SELECT JSON_FORMAT(JSON '{"a": 1, "b": "c"}')""",
|
||||
write={
|
||||
"spark": """SELECT REGEXP_EXTRACT(TO_JSON(FROM_JSON('[{"a": 1, "b": "c"}]', SCHEMA_OF_JSON('[{"a": 1, "b": "c"}]'))), '^.(.*).$', 1)""",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
"""SELECT JSON_FORMAT(JSON '[1, 2, 3]')""",
|
||||
write={
|
||||
"spark": "SELECT REGEXP_EXTRACT(TO_JSON(FROM_JSON('[[1, 2, 3]]', SCHEMA_OF_JSON('[[1, 2, 3]]'))), '^.(.*).$', 1)",
|
||||
},
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -180,7 +180,7 @@ class TestRedshift(Validator):
|
|||
"DATEDIFF('day', a, b)",
|
||||
write={
|
||||
"redshift": "DATEDIFF(day, a, b)",
|
||||
"presto": "DATE_DIFF('day', CAST(SUBSTR(CAST(a AS VARCHAR), 1, 10) AS DATE), CAST(SUBSTR(CAST(b AS VARCHAR), 1, 10) AS DATE))",
|
||||
"presto": "DATE_DIFF('day', CAST(CAST(a AS TIMESTAMP) AS DATE), CAST(CAST(b AS TIMESTAMP) AS DATE))",
|
||||
},
|
||||
)
|
||||
self.validate_all(
|
||||
|
|
|
|||
|
|
@ -575,6 +575,7 @@ class TestSnowflake(Validator):
|
|||
)
|
||||
|
||||
def test_ddl(self):
|
||||
self.validate_identity("CREATE OR REPLACE VIEW foo (uid) COPY GRANTS AS (SELECT 1)")
|
||||
self.validate_identity("CREATE TABLE geospatial_table (id INT, g GEOGRAPHY)")
|
||||
self.validate_identity("CREATE MATERIALIZED VIEW a COMMENT='...' AS SELECT 1 FROM x")
|
||||
self.validate_identity("CREATE DATABASE mytestdb_clone CLONE mytestdb")
|
||||
|
|
|
|||
|
|
@ -130,7 +130,7 @@ TBLPROPERTIES (
|
|||
write={
|
||||
"duckdb": "CAST(x AS DATE)",
|
||||
"hive": "TO_DATE(x)",
|
||||
"presto": "CAST(SUBSTR(CAST(x AS VARCHAR), 1, 10) AS DATE)",
|
||||
"presto": "CAST(CAST(x AS TIMESTAMP) AS DATE)",
|
||||
"spark": "TO_DATE(x)",
|
||||
},
|
||||
)
|
||||
|
|
@ -268,10 +268,10 @@ TBLPROPERTIES (
|
|||
write={
|
||||
"databricks": "SELECT DATEDIFF(MONTH, TO_DATE('2020-01-01'), TO_DATE('2020-03-05'))",
|
||||
"hive": "SELECT MONTHS_BETWEEN(TO_DATE('2020-03-05'), TO_DATE('2020-01-01'))",
|
||||
"presto": "SELECT DATE_DIFF('MONTH', CAST(SUBSTR(CAST('2020-01-01' AS VARCHAR), 1, 10) AS DATE), CAST(SUBSTR(CAST('2020-03-05' AS VARCHAR), 1, 10) AS DATE))",
|
||||
"presto": "SELECT DATE_DIFF('MONTH', CAST(CAST('2020-01-01' AS TIMESTAMP) AS DATE), CAST(CAST('2020-03-05' AS TIMESTAMP) AS DATE))",
|
||||
"spark": "SELECT DATEDIFF(MONTH, TO_DATE('2020-01-01'), TO_DATE('2020-03-05'))",
|
||||
"spark2": "SELECT MONTHS_BETWEEN(TO_DATE('2020-03-05'), TO_DATE('2020-01-01'))",
|
||||
"trino": "SELECT DATE_DIFF('MONTH', CAST(SUBSTR(CAST('2020-01-01' AS VARCHAR), 1, 10) AS DATE), CAST(SUBSTR(CAST('2020-03-05' AS VARCHAR), 1, 10) AS DATE))",
|
||||
"trino": "SELECT DATE_DIFF('MONTH', CAST(CAST('2020-01-01' AS TIMESTAMP) AS DATE), CAST(CAST('2020-03-05' AS TIMESTAMP) AS DATE))",
|
||||
},
|
||||
)
|
||||
|
||||
|
|
@ -359,7 +359,7 @@ TBLPROPERTIES (
|
|||
"MONTH('2021-03-01')",
|
||||
write={
|
||||
"duckdb": "MONTH(CAST('2021-03-01' AS DATE))",
|
||||
"presto": "MONTH(CAST(SUBSTR(CAST('2021-03-01' AS VARCHAR), 1, 10) AS DATE))",
|
||||
"presto": "MONTH(CAST(CAST('2021-03-01' AS TIMESTAMP) AS DATE))",
|
||||
"hive": "MONTH(TO_DATE('2021-03-01'))",
|
||||
"spark": "MONTH(TO_DATE('2021-03-01'))",
|
||||
},
|
||||
|
|
@ -368,7 +368,7 @@ TBLPROPERTIES (
|
|||
"YEAR('2021-03-01')",
|
||||
write={
|
||||
"duckdb": "YEAR(CAST('2021-03-01' AS DATE))",
|
||||
"presto": "YEAR(CAST(SUBSTR(CAST('2021-03-01' AS VARCHAR), 1, 10) AS DATE))",
|
||||
"presto": "YEAR(CAST(CAST('2021-03-01' AS TIMESTAMP) AS DATE))",
|
||||
"hive": "YEAR(TO_DATE('2021-03-01'))",
|
||||
"spark": "YEAR(TO_DATE('2021-03-01'))",
|
||||
},
|
||||
|
|
|
|||
|
|
@ -6,6 +6,8 @@ class TestTSQL(Validator):
|
|||
dialect = "tsql"
|
||||
|
||||
def test_tsql(self):
|
||||
self.validate_identity("SELECT * FROM t WITH (TABLOCK, INDEX(myindex))")
|
||||
self.validate_identity("SELECT * FROM t WITH (NOWAIT)")
|
||||
self.validate_identity("SELECT CASE WHEN a > 1 THEN b END")
|
||||
self.validate_identity("SELECT * FROM taxi ORDER BY 1 OFFSET 0 ROWS FETCH NEXT 3 ROWS ONLY")
|
||||
self.validate_identity("END")
|
||||
|
|
|
|||
2
tests/fixtures/identity.sql
vendored
2
tests/fixtures/identity.sql
vendored
|
|
@ -519,8 +519,6 @@ SELECT student, score FROM tests CROSS JOIN UNNEST(scores) AS t(a, b)
|
|||
SELECT student, score FROM tests CROSS JOIN UNNEST(scores) WITH ORDINALITY AS t(a, b)
|
||||
SELECT student, score FROM tests CROSS JOIN UNNEST(x.scores) AS t(score)
|
||||
SELECT student, score FROM tests CROSS JOIN UNNEST(ARRAY(x.scores)) AS t(score)
|
||||
SELECT * FROM t WITH (TABLOCK, INDEX(myindex))
|
||||
SELECT * FROM t WITH (NOWAIT)
|
||||
CREATE TABLE foo AS (SELECT 1) UNION ALL (SELECT 2)
|
||||
CREATE TABLE foo (id INT PRIMARY KEY ASC)
|
||||
CREATE TABLE a.b AS SELECT 1
|
||||
|
|
|
|||
35
tests/fixtures/optimizer/optimizer.sql
vendored
35
tests/fixtures/optimizer/optimizer.sql
vendored
|
|
@ -646,3 +646,38 @@ CROSS JOIN LATERAL (
|
|||
"l"."log_date" DESC NULLS LAST
|
||||
LIMIT 1
|
||||
) AS "l";
|
||||
|
||||
# title: bigquery column identifiers are case-insensitive
|
||||
# execute: false
|
||||
# dialect: bigquery
|
||||
WITH cte AS (
|
||||
SELECT
|
||||
refresh_date AS `reFREsh_date`,
|
||||
term AS `TeRm`,
|
||||
`rank`
|
||||
FROM `bigquery-public-data.GooGle_tReNDs.TOp_TeRmS`
|
||||
)
|
||||
SELECT
|
||||
refresh_date AS `Day`,
|
||||
term AS Top_Term,
|
||||
rank,
|
||||
FROM cte
|
||||
WHERE
|
||||
rank = 1
|
||||
AND refresh_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 2 WEEK)
|
||||
GROUP BY `dAy`, `top_term`, rank
|
||||
ORDER BY `DaY` DESC;
|
||||
SELECT
|
||||
`TOp_TeRmS`.`refresh_date` AS `day`,
|
||||
`TOp_TeRmS`.`term` AS `top_term`,
|
||||
`TOp_TeRmS`.`rank` AS `rank`
|
||||
FROM `bigquery-public-data`.`GooGle_tReNDs`.`TOp_TeRmS` AS `TOp_TeRmS`
|
||||
WHERE
|
||||
`TOp_TeRmS`.`rank` = 1
|
||||
AND CAST(`TOp_TeRmS`.`refresh_date` AS DATE) >= DATE_SUB(CURRENT_DATE, INTERVAL 2 WEEK)
|
||||
GROUP BY
|
||||
`TOp_TeRmS`.`refresh_date`,
|
||||
`TOp_TeRmS`.`term`,
|
||||
`TOp_TeRmS`.`rank`
|
||||
ORDER BY
|
||||
`day` DESC;
|
||||
|
|
|
|||
6
tests/fixtures/optimizer/simplify.sql
vendored
6
tests/fixtures/optimizer/simplify.sql
vendored
|
|
@ -240,6 +240,12 @@ A AND B AND C;
|
|||
SELECT x WHERE TRUE;
|
||||
SELECT x;
|
||||
|
||||
SELECT x FROM y LEFT JOIN z ON TRUE;
|
||||
SELECT x FROM y CROSS JOIN z;
|
||||
|
||||
SELECT x FROM y JOIN z USING (x);
|
||||
SELECT x FROM y JOIN z USING (x);
|
||||
|
||||
--------------------------------------
|
||||
-- Parenthesis removal
|
||||
--------------------------------------
|
||||
|
|
|
|||
|
|
@ -813,6 +813,7 @@ FROM foo""",
|
|||
self.assertEqual(
|
||||
exp.DataType.build("struct<x int>", dialect="spark").sql(), "STRUCT<x INT>"
|
||||
)
|
||||
self.assertEqual(exp.DataType.build("USER-DEFINED").sql(), "USER-DEFINED")
|
||||
|
||||
def test_rename_table(self):
|
||||
self.assertEqual(
|
||||
|
|
|
|||
|
|
@ -201,7 +201,7 @@ class TestSchema(unittest.TestCase):
|
|||
def test_schema_normalization(self):
|
||||
schema = MappingSchema(
|
||||
schema={"x": {"`y`": {"Z": {"a": "INT", "`B`": "VARCHAR"}, "w": {"C": "INT"}}}},
|
||||
dialect="spark",
|
||||
dialect="clickhouse",
|
||||
)
|
||||
|
||||
table_z = exp.Table(this="z", db="y", catalog="x")
|
||||
|
|
@ -228,4 +228,11 @@ class TestSchema(unittest.TestCase):
|
|||
|
||||
# Check that the correct dialect is used when calling schema methods
|
||||
schema = MappingSchema(schema={"[Fo]": {"x": "int"}}, dialect="tsql")
|
||||
self.assertEqual(schema.column_names("[Fo]"), schema.column_names("`Fo`", dialect="spark"))
|
||||
self.assertEqual(
|
||||
schema.column_names("[Fo]"), schema.column_names("`Fo`", dialect="clickhouse")
|
||||
)
|
||||
|
||||
# Check that all column identifiers are normalized to lowercase for BigQuery, even quoted
|
||||
# ones. Also, ensure that tables aren't normalized, since they're case-sensitive by default.
|
||||
schema = MappingSchema(schema={"Foo": {"`BaR`": "int"}}, dialect="bigquery")
|
||||
self.assertEqual(schema.column_names("Foo"), ["bar"])
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue