Skip to content

Commit

Permalink
feat: connector for Neo4j
Browse files Browse the repository at this point in the history
  • Loading branch information
keith-fullsight committed Oct 18, 2024
1 parent e95d403 commit bff830a
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 18 deletions.
15 changes: 3 additions & 12 deletions metadata-ingestion/docs/sources/neo4j/neo4j.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,10 @@
# Source Name

<!-- Set Support Status -->
![Certified](https://img.shields.io/badge/support%20status-certified-brightgreen)
![Incubating](https://img.shields.io/badge/support%20status-incubating-blue)
![Testing](https://img.shields.io/badge/support%20status-testing-lightgrey)

## Integration Details

<!-- Plain-language description of what this integration is meant to do. -->
<!-- Include details about where metadata is extracted from (ie. logs, source API, manifest, etc.) -->
Neo4j metadata will be ingested into DataHub using Call apoc.meta.data(); The data that is returned will be parsed
Neo4j metadata will be ingested into DataHub using
`CALL apoc.meta.schema() YIELD value UNWIND keys(value) AS key RETURN key, value[key] AS value;`
The data that is returned will be parsed
and will be displayed as Nodes and Relationships in DataHub. Each object will be tagged with describing what kind of DataHub
object it is. The defaults are 'Node' and 'Relationship'. These tag values can be overwritten in the recipe.

Expand All @@ -28,10 +23,6 @@ In order to ingest metadata from Neo4j, you will need:

Run the following commands to install the relevant plugin(s):

`pip install 'acryl-datahub[neo4j]'`


### Configure the Ingestion Recipe(s)

Use the following recipe(s) to get started with ingestion.

Expand Down
1 change: 0 additions & 1 deletion metadata-ingestion/docs/sources/neo4j/neo4j_recipe.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ source:
uri: 'neo4j+ssc://host:7687'
username: 'neo4j'
password: 'password'
gms_server: 'http://localhost:8080'
node_tag: 'Node'
relationship_tag: 'Relationship'
environment: 'PROD'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from neo4j import GraphDatabase
from pydantic.fields import Field

from datahub.configuration.common import ConfigModel
from datahub.configuration.source_common import EnvConfigMixin
from datahub.emitter.mce_builder import (
make_data_platform_urn,
make_dataset_urn,
Expand Down Expand Up @@ -56,11 +56,10 @@
}


class Neo4jConfig(ConfigModel):
class Neo4jConfig(EnvConfigMixin):
username: str = Field(default=None, description="Neo4j Username")
password: str = Field(default=None, description="Neo4j Password")
uri: str = Field(default=None, description="The URI for the Neo4j server")
gms_server: str = Field(default=None, description="Address for the gms server")
environment: str = Field(default=None, description="Neo4j env")
node_tag: str = Field(
default="Node",
Expand All @@ -79,7 +78,7 @@ class Neo4jSourceReport(SourceReport):
obj_created: int = 0


@platform_name("Neo4j",id="neo4j")
@platform_name("Neo4j", id="neo4j")
@config_class(Neo4jConfig)
@support_status(SupportStatus.CERTIFIED)
class Neo4jSource(Source):
Expand Down Expand Up @@ -168,7 +167,7 @@ def generate_neo4j_object(
def add_tag_to_dataset(
self, table_name: str, tag_name: str
) -> MetadataChangeProposalWrapper:
graph = DataHubGraph(DatahubClientConfig(server=self.config.gms_server))
graph = DataHubGraph(DatahubClientConfig())
dataset_urn = make_dataset_urn(
platform=self.config.platform, name=table_name, env=self.config.environment
)
Expand Down

0 comments on commit bff830a

Please sign in to comment.