Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

scql引擎连接doris2.1.7数据源查询错误 #411

Open
0xqq opened this issue Dec 2, 2024 · 3 comments · May be fixed by #413
Open

scql引擎连接doris2.1.7数据源查询错误 #411

0xqq opened this issue Dec 2, 2024 · 3 comments · May be fixed by #413
Assignees
@jingshi-ant

Comments

@0xqq
Copy link

0xqq commented Dec 2, 2024

Issue Type

Running

Have you searched for existing issues?

Yes

OS Platform and Distribution

mac os 吗

SCQL Version

0.9.1b1

What happend and What you expected to happen.

ExecuteQueryException: [320] RunExecutionPlan run jobs(82855d12-b091-11ef-b254-0242ac120006) failed, catch std::exception=[Enforce fail at engine/datasource/datasource_adaptor.cc:29] tensors.size() == expected_outputs.size(). 0 vs 4.query result column size=0 not equal to expected size=4; (peer: bob, code: 320, msg: RunExecutionPlan run jobs(82855d12-b091-11ef-b254-0242ac120006) failed, catch std::exception=[Enforce fail at engine/datasource/datasource_adaptor.cc:29] tensors.size() == expected_outputs.size(). 0 vs 3.query result column size=0 not equal to expected size=3)

Configuration used to run SCQL.

SELECT
  ta.credit_rank,
  COUNT(*) AS cnt,
  AVG(ta.income) AS avg_income,
  AVG(tb.order_amount) as avg_amount
FROM
  ta
  INNER JOIN tb ON ta.ID = tb.ID
WHERE
  ta.age >= 20
  AND ta.age <= 30
  AND tb.is_active = 1
GROUP BY
  ta.credit_rank

SCQL log output.

2024-12-02 09:17:40.1229 INFO checksum.go:103 get local checksum table schema: [212 29 236 61 47 240 204 56 147 38];ccl: [160 138 174 88 19 7 59 219 47 186] for party alice
2024-12-02 09:17:41.1229 INFO executor.go:152 |RequestID:|SessionID:47dcc4c0-b08e-11ef-b254-0242ac120006|ActionName:Executor@RunExecutionPlan|CostTime:145.904042ms|Reason:|ErrorMsg:|Request:{"job_params":{"party_code":"alice", "parties":[{"code":"alice", "name":"alice", "host":"engine_alice:8003", "public_key":"MCowBQYDK2VwAyEAu44BqfHC+T7VXUqlEi2OWSk4F1QANkbFRSnVz4DGrOU="}, {"code":"bob", "name":"bob", "host":"engine_bob:8004", "rank":1, "public_key":"MCowBQYDK2VwAyEAJSvMGdqvoYqQuu8fVjXK+potV4Ee2tZFq2nMXei/nsw="}], "job_id":"47dcc4c0-b08e-11ef-b254-0242ac120006", "spu_runtime_cfg":{"protocol":"SEMI2K", "field":"FM64"}, "link_cfg":{}, "psi_cfg":{}, "log_cfg":{}}, "graph":{"nodes":{"0":{"node_name":"runsql.0", "op_type":"RunSQL", "outputs":{"Out":{"tensors":[{"name":"alice.ta.ID.0", "elem_type":"STRING", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":2}, {"name":"alice.ta.age.1", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}, {"name":"alice.ta.credit_rank.2", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}, {"name":"alice.ta.income.3", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "attributes":{"sql":{"t":{"name":".0", "elem_type":"STRING", "string_data":["select user_credit.ID,user_credit.age,user_credit.credit_rank,user_credit.income from alice.user_credit where (user_credit.age>=20) and (user_credit.age<=30)"]}}, "table_refs":{"t":{"name":".0", "shape":{"dim":[{"dim_value":"1"}]}, "elem_type":"STRING", "string_data":["alice.user_credit"]}}}}, "10":{"node_name":"make_share.10", "op_type":"MakeShare", "inputs":{"In":{"tensors":[{"name":"alice.tb.is_active.14", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.tb.is_active.21", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}}, "11":{"node_name":"make_share.11", "op_type":"MakeShare", "inputs":{"In":{"tensors":[{"name":"alice.tb.order_amount.15", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.tb.order_amount.22", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}}, "12":{"node_name":"sort.12", "op_type":"Sort", "inputs":{"In":{"tensors":[{"name":"alice.ta.credit_rank.16", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":3}, {"name":"alice.ta.ID.17", "elem_type":"STRING", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.ta.age.18", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.ta.credit_rank.16", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":3}, {"name":"alice.ta.income.19", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.tb.ID.20", "elem_type":"STRING", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.tb.is_active.21", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.tb.order_amount.22", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}, "Key":{"tensors":[{"name":"alice.ta.credit_rank.16", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":3}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.ta.credit_rank.23", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.ta.ID.24", "elem_type":"STRING", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}}, {"name":"alice.ta.age.25", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}}, {"name":"alice.ta.credit_rank.26", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.ta.income.27", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.tb.ID.28", "elem_type":"STRING", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}}, {"name":"alice.tb.is_active.29", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}}, {"name":"alice.tb.order_amount.30", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}, "attributes":{"reverse":{"t":{"name":".0", "elem_type":"BOOL", "bool_data":[false]}}}}, "13":{"node_name":"group_mark.13", "op_type":"ObliviousGroupMark", "inputs":{"Key":{"tensors":[{"name":"alice.ta.credit_rank.23", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}, "outputs":{"Group":{"tensors":[{"name":"group_mark.31", "elem_type":"BOOL", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":5}]}}}, "14":{"node_name":"count.14", "op_type":"ObliviousGroupCount", "inputs":{"Group":{"tensors":[{"name":"group_mark.31", "elem_type":"BOOL", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":5}]}, "In":{"tensors":[{"name":"group_mark.31", "elem_type":"BOOL", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":5}]}}, "outputs":{"Out":{"tensors":[{"name":"group_mark.32", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}}, "15":{"node_name":"avg.15", "op_type":"ObliviousGroupAvg", "inputs":{"Group":{"tensors":[{"name":"group_mark.31", "elem_type":"BOOL", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":5}]}, "In":{"tensors":[{"name":"alice.ta.income.27", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.ta.income.33", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}}, "16":{"node_name":"avg.16", "op_type":"ObliviousGroupAvg", "inputs":{"Group":{"tensors":[{"name":"group_mark.31", "elem_type":"BOOL", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":5}]}, "In":{"tensors":[{"name":"alice.tb.order_amount.30", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.tb.order_amount.34", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}}, "17":{"node_name":"shuffle.17", "op_type":"Shuffle", "inputs":{"In":{"tensors":[{"name":"group_mark.32", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.ta.income.33", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.tb.order_amount.34", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.ta.credit_rank.26", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"group_mark.31", "elem_type":"BOOL", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":5}]}}, "outputs":{"Out":{"tensors":[{"name":"group_mark.35", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.ta.income.36", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.tb.order_amount.37", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.ta.credit_rank.38", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"group_mark.39", "elem_type":"BOOL", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}}, "18":{"node_name":"make_public.18", "op_type":"MakePublic", "inputs":{"In":{"tensors":[{"name":"group_mark.39", "elem_type":"BOOL", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"group_mark.40", "elem_type":"BOOL", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PUBLIC"}, "ref_num":1}]}}}, "19":{"node_name":"filter.19", "op_type":"Filter", "inputs":{"Filter":{"tensors":[{"name":"group_mark.40", "elem_type":"BOOL", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PUBLIC"}, "ref_num":1}]}, "In":{"tensors":[{"name":"group_mark.35", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.ta.income.36", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.tb.order_amount.37", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.ta.credit_rank.38", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"group_mark.41", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":3}, {"name":"alice.ta.income.42", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.tb.order_amount.43", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.ta.credit_rank.44", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}}, "2":{"node_name":"join.2", "op_type":"Join", "inputs":{"Left":{"tensors":[{"name":"alice.ta.ID.0", "elem_type":"STRING", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":2}]}, "Right":{"tensors":[{"name":"alice.tb.ID.4", "elem_type":"STRING", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":2}]}}, "outputs":{"LeftJoinIndex":{"tensors":[{"name":"alice.ta.ID.7", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}, "RightJoinIndex":{"tensors":[{"name":"alice.tb.ID.8", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "attributes":{"input_party_codes":{"t":{"name":".0", "shape":{"dim":[{"dim_value":"2"}]}, "elem_type":"STRING", "string_data":["alice", "bob"]}}, "join_type":{"t":{"name":".0", "elem_type":"INT64", "int64_data":["0"]}}, "psi_algorithm":{"t":{"name":".0", "elem_type":"INT64", "int64_data":["0"]}}}}, "20":{"node_name":"make_constant.20", "op_type":"Constant", "outputs":{"Out":{"tensors":[{"name":"constant_data.45", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PUBLIC"}, "ref_num":1}]}}, "attributes":{"scalar":{"t":{"name":".0", "elem_type":"INT64", "int64_data":["4"]}}, "to_status":{"t":{"name":".0", "elem_type":"INT64", "int64_data":["1"]}}}}, "21":{"node_name":"broadcast.21", "op_type":"BroadcastTo", "inputs":{"In":{"tensors":[{"name":"constant_data.45", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PUBLIC"}, "ref_num":1}]}, "ShapeRefTensor":{"tensors":[{"name":"group_mark.41", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":3}]}}, "outputs":{"Out":{"tensors":[{"name":"constant_data.46", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PUBLIC"}, "ref_num":1}]}}}, "22":{"node_name":"make_private.22", "op_type":"MakePrivate", "inputs":{"In":{"tensors":[{"name":"group_mark.41", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":3}]}}, "outputs":{"Out":{"tensors":[{"name":"group_mark.47", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "attributes":{"reveal_to":{"t":{"name":".0", "elem_type":"STRING", "string_data":["alice"]}}}}, "23":{"node_name":"make_private.23", "op_type":"MakePrivate", "inputs":{"In":{"tensors":[{"name":"constant_data.46", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PUBLIC"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"constant_data.48", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "attributes":{"reveal_to":{"t":{"name":".0", "elem_type":"STRING", "string_data":["alice"]}}}}, "24":{"node_name":"GreaterEqual.24", "op_type":"GreaterEqual", "inputs":{"Left":{"tensors":[{"name":"group_mark.47", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}, "Right":{"tensors":[{"name":"constant_data.48", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"GreaterEqual_out.49", "elem_type":"BOOL", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}}, "25":{"node_name":"make_public.25", "op_type":"MakePublic", "inputs":{"In":{"tensors":[{"name":"GreaterEqual_out.49", "elem_type":"BOOL", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"GreaterEqual_out.50", "elem_type":"BOOL", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PUBLIC"}, "ref_num":1}]}}}, "26":{"node_name":"apply_filter.26", "op_type":"Filter", "inputs":{"Filter":{"tensors":[{"name":"GreaterEqual_out.50", "elem_type":"BOOL", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PUBLIC"}, "ref_num":1}]}, "In":{"tensors":[{"name":"alice.ta.credit_rank.44", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"group_mark.41", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":3}, {"name":"alice.ta.income.42", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.tb.order_amount.43", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.ta.credit_rank.51", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"group_mark.52", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.ta.income.53", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}, {"name":"alice.tb.order_amount.54", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}}, "27":{"node_name":"make_private.27", "op_type":"MakePrivate", "inputs":{"In":{"tensors":[{"name":"alice.ta.credit_rank.51", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.ta.credit_rank.55", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "attributes":{"reveal_to":{"t":{"name":".0", "elem_type":"STRING", "string_data":["alice"]}}}}, "28":{"node_name":"make_private.28", "op_type":"MakePrivate", "inputs":{"In":{"tensors":[{"name":"group_mark.52", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"group_mark.57", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "attributes":{"reveal_to":{"t":{"name":".0", "elem_type":"STRING", "string_data":["alice"]}}}}, "29":{"node_name":"make_private.29", "op_type":"MakePrivate", "inputs":{"In":{"tensors":[{"name":"alice.ta.income.53", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.ta.income.59", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "attributes":{"reveal_to":{"t":{"name":".0", "elem_type":"STRING", "string_data":["alice"]}}}}, "3":{"node_name":"filter_by_index.3", "op_type":"FilterByIndex", "inputs":{"Data":{"tensors":[{"name":"alice.ta.ID.0", "elem_type":"STRING", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":2}, {"name":"alice.ta.age.1", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}, {"name":"alice.ta.credit_rank.2", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}, {"name":"alice.ta.income.3", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}, "RowsIndexFilter":{"tensors":[{"name":"alice.ta.ID.7", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.ta.ID.9", "elem_type":"STRING", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}, {"name":"alice.ta.age.10", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}, {"name":"alice.ta.credit_rank.11", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}, {"name":"alice.ta.income.12", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}}, "30":{"node_name":"make_private.30", "op_type":"MakePrivate", "inputs":{"In":{"tensors":[{"name":"alice.tb.order_amount.54", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.tb.order_amount.61", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "attributes":{"reveal_to":{"t":{"name":".0", "elem_type":"STRING", "string_data":["alice"]}}}}, "31":{"node_name":"publish.31", "op_type":"Publish", "inputs":{"In":{"tensors":[{"name":"alice.ta.credit_rank.55", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}, {"name":"group_mark.57", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}, {"name":"alice.ta.income.59", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}, {"name":"alice.tb.order_amount.61", "elem_type":"FLOAT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"credit_rank.56", "elem_type":"STRING", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "string_data":["credit_rank"]}, {"name":"cnt.58", "elem_type":"STRING", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "string_data":["cnt"]}, {"name":"avg_income.60", "elem_type":"STRING", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "string_data":["avg_income"]}, {"name":"avg_amount.62", "elem_type":"STRING", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "string_data":["avg_amount"]}]}}}, "5":{"node_name":"make_share.5", "op_type":"MakeShare", "inputs":{"In":{"tensors":[{"name":"alice.ta.credit_rank.11", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.ta.credit_rank.16", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":3}]}}}, "6":{"node_name":"make_share.6", "op_type":"MakeShare", "inputs":{"In":{"tensors":[{"name":"alice.ta.ID.9", "elem_type":"STRING", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.ta.ID.17", "elem_type":"STRING", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}}, "7":{"node_name":"make_share.7", "op_type":"MakeShare", "inputs":{"In":{"tensors":[{"name":"alice.ta.age.10", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.ta.age.18", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}}, "8":{"node_name":"make_share.8", "op_type":"MakeShare", "inputs":{"In":{"tensors":[{"name":"alice.ta.income.12", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.ta.income.19", "elem_type":"INT64", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}}, "9":{"node_name":"make_share.9", "op_type":"MakeShare", "inputs":{"In":{"tensors":[{"name":"alice.tb.ID.13", "elem_type":"STRING", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_PRIVATE"}, "ref_num":1}]}}, "outputs":{"Out":{"tensors":[{"name":"alice.tb.ID.20", "elem_type":"STRING", "option":"REFERENCE", "annotation":{"status":"TENSORSTATUS_SECRET"}, "ref_num":1}]}}}}, "policy":{"worker_num":1, "pipelines":[{"subdags":[{"jobs":[{"node_ids":["0"]}]}, {}, {"jobs":[{"node_ids":["20"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["2"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["3"]}]}, {}, {"jobs":[{"node_ids":["5"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["6"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["7"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["8"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["9"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["10"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["11"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["12"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["13"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["14"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["15"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["16"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["17"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["18"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["19"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["21"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["22"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["23"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["24"]}]}, {"jobs":[{"node_ids":["25"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["26"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["27"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["28"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["29"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["30"]}], "need_call_barrier_after_jobs":true}, {"jobs":[{"node_ids":["31"]}]}]}]}, "sub_graph_checksum":"2ff57ff047699d25beaadf6d28e30f825596f3e510f773027908f067fdad9c65"}, "graph_checksum":{"check_graph_checksum":true, "sub_graph_checksums":{"0":"2ff57ff047699d25beaadf6d28e30f825596f3e510f773027908f067fdad9c65", "1":"ce6d980b971bec8955f72988e682d2c5749675ad94d6a2e9f44a1fca044a7523"}, "whole_graph_checksum":"7bd60f1fe83284740e1c6ae2a72df46ce7f596b4d076622db118a77e9ad83656"}, "callback_url":"http://broker_alice:8080/intra/cb/engine"}|PartyCode:alice|Url:engine_alice:8003
2024-12-02 09:17:41.1229 ERROR executor.go:93 |RequestID:|SessionID:47dcc4c0-b08e-11ef-b254-0242ac120006|ActionName:EngineStub@RunExecutionPlan|CostTime:147.448708ms|Reason:InvalidResponse|ErrorMsg:Error: code=320, msg="RunExecutionPlan run jobs(47dcc4c0-b08e-11ef-b254-0242ac120006) failed, catch std::exception=[Enforce fail at engine/datasource/datasource_adaptor.cc:29] tensors.size() == expected_outputs.size(). 0 vs 4.query result column size=0 not equal to expected size=4"|Request:
2024-12-02 09:17:41.1229 ERROR common.go:148 |RequestID:|RequestParty:|SessionID:|ActionName:Intra@DoQuery|CostTime:191.798291ms|Reason:|ErrorMsg:runQuery Execute err: Error: code=320, msg="RunExecutionPlan run jobs(47dcc4c0-b08e-11ef-b254-0242ac120006) failed, catch std::exception=[Enforce fail at engine/datasource/datasource_adaptor.cc:29] tensors.size() == expected_outputs.size(). 0 vs 4.query result column size=0 not equal to expected size=4"|Request:project_id:"alice"  query:"SELECT\n  ta.credit_rank,\n  COUNT(*) AS cnt,\n  AVG(ta.income) AS avg_income,\n  AVG(tb.order_amount) as avg_amount\nFROM\n  ta\n  INNER JOIN tb ON ta.ID = tb.ID\nWHERE\n  ta.age >= 20\n  AND ta.age <= 30\n  AND tb.is_active = 1\nGROUP BY\n  ta.credit_rank"
2024-12-02 09:17:41.1229 INFO server.go:136 |GIN|status=200|method=POST|path=/intra/query|ip=172.18.0.1|latency=192.234125ms|
@jingshi-ant
Copy link
Collaborator

您好，能否提供下复现环境的步骤？包括：doris的参考部署流程；engine的gflags.conf；

@0xqq
Copy link
Author

0xqq commented Dec 3, 2024

alice gflags.conf
--listen_port=8003
--enable_separate_link_port=true
--link_port=8004
--datasource_router=embed
--enable_driver_authorization=false
--embed_router_conf={"datasources":[{"id":"ds001","name":"mysql db","kind":"MYSQL","connection_str":"db=alice;user=scql;password=1;host=192.168.96.1;port=9030;auto-reconnect=true"}],"rules":[{"db":"","table":"","datasource_id":"ds001"}]}

party authentication flags

--enable_self_auth=false
--enable_peer_auth=false

https flags

--server_enable_ssl=true
--server_ssl_certificate=/home/admin/engine/conf/cert.crt
--server_ssl_private_key=/home/admin/engine/conf/key.key

set peer_engine_enable_ssl_as_client to true when peer SCQLEngine has https enabled

--peer_engine_enable_ssl_as_client=true

set false when SCQLBroker IntraServer not enable https

--driver_enable_ssl_as_client=false

bob gflags.conf

--listen_port=8003
--enable_separate_link_port=true
--link_port=8004
--datasource_router=embed
--enable_driver_authorization=false
--embed_router_conf={"datasources":[{"id":"ds001","name":"mysql db","kind":"MYSQL","connection_str":"db=bob;user=scql;password=1;host=192.168.96.1;port=9030;auto-reconnect=true"}],"rules":[{"db":"","table":"","datasource_id":"ds001"}]}

party authentication flags

--enable_self_auth=false
--enable_peer_auth=false

https flags

--server_enable_ssl=true
--server_ssl_certificate=/home/admin/engine/conf/cert.crt
--server_ssl_private_key=/home/admin/engine/conf/key.key

set peer_engine_enable_ssl_as_client to true when peer SCQLEngine has https enabled

--peer_engine_enable_ssl_as_client=true

set false when SCQLBroker IntraServer not enable https

--driver_enable_ssl_as_client=false

@0xqq
Copy link
Author

0xqq commented Dec 3, 2024

doris2.1.7 sql脚本：

create database alice;

use alice;
CREATE TABLE user_credit (
ID varchar(64) NOT NULL,
credit_rank int NOT NULL,
income int NOT NULL,
age int NOT NULL
) ENGINE=OLAP
DUPLICATE KEY(ID)
COMMENT 'alice'
DISTRIBUTED BY HASH(ID) BUCKETS AUTO
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"min_load_replica_num" = "-1",
"is_being_synced" = "false",
"storage_medium" = "hdd",
"storage_format" = "V2",
"inverted_index_storage_format" = "V1",
"light_schema_change" = "true",
"store_row_column" = "true",
"row_store_page_size" = "16384",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false",
"group_commit_interval_ms" = "10000",
"group_commit_data_bytes" = "134217728",
"enable_mow_light_delete" = "false"
);

INSERT INTO user_credit VALUES ("id0001", 6, 100000, 20);
INSERT INTO user_credit VALUES ("id0002", 5, 90000, 19);
INSERT INTO user_credit VALUES ("id0003", 6, 89700, 32);
INSERT INTO user_credit VALUES ("id0005", 6, 607000, 30);
INSERT INTO user_credit VALUES ("id0006", 5, 30070, 25);
INSERT INTO user_credit VALUES ("id0007", 6, 12070, 28);
INSERT INTO user_credit VALUES ("id0008", 6, 200800, 50);
INSERT INTO user_credit VALUES ("id0009", 6, 607000, 30);
INSERT INTO user_credit VALUES ("id0010", 5, 30070, 25);
INSERT INTO user_credit VALUES ("id0011", 5, 12070, 28);
INSERT INTO user_credit VALUES ("id0012", 6, 200800, 50);
INSERT INTO user_credit VALUES ("id0013", 5, 30070, 25);
INSERT INTO user_credit VALUES ("id0014", 5, 12070, 28);
INSERT INTO user_credit VALUES ("id0015", 6, 200800, 18);
INSERT INTO user_credit VALUES ("id0016", 5, 30070, 26);
INSERT INTO user_credit VALUES ("id0017", 5, 12070, 27);
INSERT INTO user_credit VALUES ("id0018", 6, 200800, 16);
INSERT INTO user_credit VALUES ("id0019", 6, 30070, 25);
INSERT INTO user_credit VALUES ("id0020", 5, 12070, 28);

create database bob;
use bob;

CREATE TABLE user_stats (
ID varchar(64) not NULL,
order_amount float not null,
is_active tinyint(1) not null
) ENGINE=OLAP
DUPLICATE KEY(ID)
COMMENT 'alice'
DISTRIBUTED BY HASH(ID) BUCKETS AUTO
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"min_load_replica_num" = "-1",
"is_being_synced" = "false",
"storage_medium" = "hdd",
"storage_format" = "V2",
"inverted_index_storage_format" = "V1",
"light_schema_change" = "true",
"store_row_column" = "true",
"row_store_page_size" = "16384",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false",
"group_commit_interval_ms" = "10000",
"group_commit_data_bytes" = "134217728",
"enable_mow_light_delete" = "false"
);

INSERT INTO user_stats VALUES ("id0001", 3598.0, 1);
INSERT INTO user_stats VALUES ("id0002", 100.0, 0);
INSERT INTO user_stats VALUES ("id0003", 2549.0, 1);
INSERT INTO user_stats VALUES ("id0004", 21698.5, 1);
INSERT INTO user_stats VALUES ("id0005", 4985.5, 1);
INSERT INTO user_stats VALUES ("id0006", 3598.0, 1);
INSERT INTO user_stats VALUES ("id0007", 322, 0);
INSERT INTO user_stats VALUES ("id0008", 9816.2, 1);
INSERT INTO user_stats VALUES ("id0009", 3598.0, 1);
INSERT INTO user_stats VALUES ("id0010", 322, 0);
INSERT INTO user_stats VALUES ("id0011", 9816.2, 1);
INSERT INTO user_stats VALUES ("id0012", 3598.0, 1);
INSERT INTO user_stats VALUES ("id0013", 322, 0);
INSERT INTO user_stats VALUES ("id0014", 9816.2, 1);
INSERT INTO user_stats VALUES ("id0015", 9816.2, 1);
INSERT INTO user_stats VALUES ("id0016", 9816.2, 1);
INSERT INTO user_stats VALUES ("id0017", 3598.0, 1);
INSERT INTO user_stats VALUES ("id0018", 322, 0);
INSERT INTO user_stats VALUES ("id0019", 9816.2, 1);
INSERT INTO user_stats VALUES ("id0020", 9816.2, 1);

@tongke6 tongke6 linked a pull request Dec 5, 2024 that will close this issue
fix poco to connect to doris #413
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jingshi-ant jingshi-ant

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix poco to connect to doris secretflow/scql
2 participants
@0xqq @jingshi-ant