Skip to content

Commit

Permalink
Fixed invalid DAX query in LoadColumnStatistics with low columnBatchS…
Browse files Browse the repository at this point in the history
…ize (#138)

Fixed an issue where a low columnBatchSize could generate an invalid DAX query during column statistics analysis, due to filters applied to the RowNumber and GroupByColumn properties.
  • Loading branch information
albertospelta authored Aug 9, 2024
1 parent f8ae1f1 commit e838ddc
Showing 1 changed file with 3 additions and 5 deletions.
8 changes: 3 additions & 5 deletions src/Dax.Model.Extractor/StatExtractor.cs
Original file line number Diff line number Diff line change
Expand Up @@ -257,7 +257,7 @@ private void LoadColumnStatistics(bool analyzeDirectQuery, DirectLakeExtractionM
// skip direct query tables if the analyzeDirectQuery is false
where t.Columns.Count > 1 && (analyzeDirectQuery || !t.HasDirectQueryPartitions)
from c in t.Columns
where c.State == "Ready"
where c.State == "Ready" && !c.IsRowNumber && c.GroupByColumns.Count == 0
// only include the column if the table does not have Direct Lake partitions or if they are resident or if analyzeDirectLake is true
&& (!t.HasDirectLakePartitions
|| (analyzeDirectLake >= DirectLakeExtractionMode.ResidentOnly && c.IsResident)
Expand All @@ -269,14 +269,12 @@ from c in t.Columns
foreach ( var columnSet in loopColumns ) {
var idString = 0;
var dax = "EVALUATE ";
int validColumns = columnSet.Where(c => !c.IsRowNumber).Count();
//only union if there is more than 1 column in the columnSet
if (validColumns > 1) { dax += "UNION("; }
if (columnSet.Count > 1) { dax += "UNION("; }
dax += string.Join(",", columnSet
.Where(c => !c.IsRowNumber && c.GroupByColumns.Count == 0 )
.Select(c => $"\n ROW(\"Table\", \"{idString++:0000}{EmbedNameInString(c.Table.TableName.Name)}\", \"Column\", \"{idString++:0000}{EmbedNameInString(c.ColumnName.Name)}\", \"Cardinality\", {DistinctCountExpression(c)})").ToList());
//only close the union call if there is more than 1 column in the columnSet
if (validColumns > 1) { dax += ")"; }
if (columnSet.Count > 1) { dax += ")"; }

var cmd = CreateCommand(dax);
cmd.CommandTimeout = CommandTimeout;
Expand Down

0 comments on commit e838ddc

Please sign in to comment.