Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Map data should be a non-nullable struct type #112

Open
tanejagagan opened this issue Dec 13, 2024 · 0 comments
Open

Map data should be a non-nullable struct type #112

tanejagagan opened this issue Dec 13, 2024 · 0 comments

Comments

@tanejagagan
Copy link

Getting exception while trying to access map data type
java.lang.IllegalArgumentException: Map data should be a non-nullable struct type

at org.apache.arrow.util.Preconditions.checkArgument(Preconditions.java:126)
at org.apache.arrow.vector.complex.MapVector.initializeChildrenFromFields(MapVector.java:96)
at org.apache.arrow.vector.types.pojo.Field.createVector(Field.java:107)
at org.apache.arrow.vector.ipc.ArrowReader.initialize(ArrowReader.java:187)
at org.apache.arrow.vector.ipc.ArrowReader.ensureInitialized(ArrowReader.java:171)
at org.apache.arrow.vector.ipc.ArrowReader.getVectorSchemaRoot(ArrowReader.java:68)
at org.apache.arrow.c.ArrowArrayStreamReader.loadNextBatch(ArrowArrayStreamReader.java:68)
at io.dazzleduck.spark.MapTypeTest.testMapType(MapTypeTest.java:28)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

import org.apache.arrow.memory.RootAllocator;
import org.apache.arrow.vector.ipc.ArrowReader;
import org.duckdb.DuckDBConnection;
import org.duckdb.DuckDBResultSet;
import org.junit.jupiter.api.Test;

import java.io.IOException;
import java.sql.DriverManager;
import java.sql.SQLException;

public class MapTypeTest {
    @Test
    void testMapType() throws SQLException, IOException {
        var selectStatement = "select map { 'k1' : 'v1' } as m";
        DuckDBConnection connection = (DuckDBConnection) DriverManager.getConnection("jdbc:duckdb:");
        try (var s = connection.createStatement()) {
            s.execute(selectStatement);
            try(var resultSet = (DuckDBResultSet)s.getResultSet();
                var allocator = new RootAllocator()) {
                    try (var reader = (ArrowReader) resultSet.arrowExportStream(allocator, 256)) {
                        while (reader.loadNextBatch()) {
                            System.out.println(reader.getVectorSchemaRoot().contentToTSVString());
                        }
                    }
            }
        }
    }
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant