Amundsen is unable to import MYSQL data

[MalavikaN1]

Expected Behavior

Changed the connection string in https://github.com/amundsen-io/amundsen/blob/main/databuilder/example/scripts/sample_mysql_loader.py to load locally hosted MySQL data into Amundsen .
Changes made in above file:

import pymysql
pymysql.install_as_MySQLdb()

def connection_string():
    user = 'root'
    password='root'
    host = 'localhost'
    port = '3307'
    db = 'test_db'
    return "mysql+pymysql://%s:%s@%s:%s/%s" % (user,password, host, port, db)

Current Behavior

While running the python file, I get the following error:
ERROR:neo4j:Failed to write data to connection IPv4Address(('127.0.0.1', 7687)) (IPv4Address(('127.0.0.1', 7687))).

I tried loading the sample data by running the https://github.com/amundsen-io/amundsen/blob/main/databuilder/example/scripts/sample_data_loader.py file and it worked.

Possible Solution

fix: Adding the below code to job_config in sample_mysql_loader.py fixed the issue.
f'publisher.neo4j.{neo4j_csv_publisher.NEO4J_ENCRYPTED}': False
So now the code looks like this:

job_config = ConfigFactory.from_dict({
        f'extractor.mysql_metadata.{MysqlMetadataExtractor.WHERE_CLAUSE_SUFFIX_KEY}': where_clause_suffix,
        f'extractor.mysql_metadata.{MysqlMetadataExtractor.USE_CATALOG_AS_CLUSTER_NAME}': True,
        f'extractor.mysql_metadata.extractor.sqlalchemy.{SQLAlchemyExtractor.CONN_STRING}': connection_string(),
        f'loader.filesystem_csv_neo4j.{FsNeo4jCSVLoader.NODE_DIR_PATH}': node_files_folder,
        f'loader.filesystem_csv_neo4j.{FsNeo4jCSVLoader.RELATION_DIR_PATH}': relationship_files_folder,
        f'publisher.neo4j.{neo4j_csv_publisher.NODE_FILES_DIR}': node_files_folder,
        f'publisher.neo4j.{neo4j_csv_publisher.RELATION_FILES_DIR}': relationship_files_folder,
        f'publisher.neo4j.{neo4j_csv_publisher.NEO4J_END_POINT_KEY}': neo4j_endpoint,
        f'publisher.neo4j.{neo4j_csv_publisher.NEO4J_USER}': neo4j_user,
        f'publisher.neo4j.{neo4j_csv_publisher.NEO4J_PASSWORD}': neo4j_password,
        f'publisher.neo4j.{neo4j_csv_publisher.NEO4J_ENCRYPTED}': False,
        f'publisher.neo4j.{neo4j_csv_publisher.JOB_PUBLISH_TAG}': 'unique_tag',  # should use unique tag here like {ds}
    })

Your Environment

• Amundsen databuilder version used: 7.4.3
• Deployment (k8s or native): native
• Link to your fork or repository: (https://github.com/MalavikaN1/amundsen)

[boring-cyborg]

Thanks for opening your first issue here!

[danielemansillo]

I am not totally sure this will help you but it worked for me.
Since

from 4.0 onwards, the default encryption setting is off

Configure SSL Policy for Bolt server and HTTPS server

I would suggest you to try to set encrypted to false in the default security configuration:

default_security_conf = {'trust': neo4j.TRUST_ALL_CERTIFICATES, 'encrypted': False}

both in databuilder/databuilder/publisher/neo4j_csv_publisher.py and databuilder/databuilder/extractor/neo4j_extractor.py

Try it out and let me know!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[stale]

This issue has been automatically closed for inactivity. If you still wish to make these changes, please open a new pull request or reopen this one.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[badsector998]

f'publisher.neo4j.{neo4j_csv_publisher.NEO4J_ENCRYPTED}': False

on possibles solutions proposed by the op worked for me.