Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Algorithm Result object complete rewrite, Add betweeen centrality, Add Zaks Karate Club #1327

Merged
merged 51 commits into from
Oct 19, 2023
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
ead4c97
checkpoint, need to fix tests
Haaroon Oct 9, 2023
e96e4bd
im so happy it works :D
Haaroon Oct 9, 2023
ee7943d
remove type_name param
Haaroon Oct 9, 2023
82e680c
algorithm result tests complete, new fmt, fixed group_by, fixed all t…
Haaroon Oct 10, 2023
1c1e6c3
renamed old algorithm result, changed base result type to a hashmap i…
Haaroon Oct 10, 2023
3d98c9c
fix bad 0 checking
Haaroon Oct 10, 2023
abe4669
macro magic but partially implemented
Haaroon Oct 11, 2023
cd6a487
added the extra macros, added all missing functions, removed all poin…
Haaroon Oct 11, 2023
23e17e9
algorithm result object + all macros is complete in both rust and python
Haaroon Oct 11, 2023
d4e89ca
fixed reciprocity, fixed centrality
Haaroon Oct 11, 2023
f6c8cd7
fixed pagerank, fixed graphql new type issues
Haaroon Oct 11, 2023
f51d45b
balance algorithm complete
Haaroon Oct 12, 2023
db3cf72
sssp fixed
Haaroon Oct 12, 2023
316137d
sssp forgot le lib
Haaroon Oct 12, 2023
43992dd
temporal reachability fixed
Haaroon Oct 12, 2023
a522719
hits fixed
Haaroon Oct 12, 2023
6860632
connected components nearly fixed, last test
Haaroon Oct 12, 2023
af7c4f2
removed bad test
Haaroon Oct 12, 2023
b49f397
three node motif completed, algorithm result complte
Haaroon Oct 12, 2023
e680e56
good bye bugs
Haaroon Oct 12, 2023
c56543e
removed all warnings
Haaroon Oct 12, 2023
0277712
fix lotr bug
Haaroon Oct 12, 2023
bb0284f
i think i fixed hulong
Haaroon Oct 12, 2023
a5d8e98
fix dusty benchmark
Haaroon Oct 12, 2023
bec7cdd
rename macro rules, fix doctests
Haaroon Oct 12, 2023
3b5a5f1
implement debug for algo result
Haaroon Oct 12, 2023
1f0a35e
fix pytests
Haaroon Oct 12, 2023
1f348f9
Zaks Karate Club Graph (#1326)
Haaroon Oct 13, 2023
8eb8959
Merge branch 'master' into feature/vertex-id-algo-v3
Haaroon Oct 13, 2023
dfdc50a
Changed State to use Internal ID everywhere, fixed most of the algori…
Haaroon Oct 16, 2023
b628300
fixed all the algos
Haaroon Oct 16, 2023
2c03156
fn name changes
Haaroon Oct 17, 2023
93a6599
added very basic hashing to allow get_all to return a vertex object
Haaroon Oct 17, 2023
8855ea7
vertex view now returned by algo rest
Haaroon Oct 17, 2023
3597c5c
fix python algo result for vertexview
Haaroon Oct 17, 2023
c68a2a4
reorder tests
Haaroon Oct 17, 2023
cc92023
implemented custom debug fmt display for vertex view, implemented cus…
Haaroon Oct 17, 2023
c472fc4
extended richcmp for python objects, fixed all python tests
Haaroon Oct 17, 2023
b2c2e1c
Merge branch 'master' into feature/vertex-id-algo-v3
Haaroon Oct 17, 2023
fb4eb05
connected_components.rs passes
Haaroon Oct 17, 2023
3fdc430
Merge remote-tracking branch 'origin/feature/vertex-id-algo-v3' into …
Haaroon Oct 17, 2023
b57be1a
Merge branch 'master' into feature/vertex-id-algo-v3
Haaroon Oct 17, 2023
28147d9
Merge branch 'master' into feature/vertex-id-algo-v3
Haaroon Oct 18, 2023
53482f9
moved algos
Haaroon Oct 18, 2023
203d6fe
resolved comments
Haaroon Oct 18, 2023
12606d2
changes due to comments
Haaroon Oct 18, 2023
ad24ca7
windows works, layers dont
Haaroon Oct 18, 2023
bfa0993
added tests for windowed graphs
Haaroon Oct 18, 2023
0abdd18
complete!
Haaroon Oct 18, 2023
c4b0a11
bad comments for ubuntu
Haaroon Oct 18, 2023
44918c8
bad colon
Haaroon Oct 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions comparison-benchmark/rust/raphtory-rust-benchmark/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -197,14 +197,12 @@ fn main() {

// page rank with time
now = Instant::now();
let _page_rank: Vec<_> = unweighted_page_rank(&g, 1000, None, None, true)
.into_iter()
.collect();
let _page_rank = unweighted_page_rank(&g, 1000, None, None, true);
println!("Page rank: {} seconds", now.elapsed().as_secs_f64());

// connected community_detection with time
now = Instant::now();
let _cc: AlgorithmResult<String, u64> = weakly_connected_components(&g, usize::MAX, None);
let _cc = weakly_connected_components(&g, usize::MAX, None);
println!(
"Connected community_detection: {} seconds",
now.elapsed().as_secs_f64()
Expand Down
1 change: 1 addition & 0 deletions docs/source/reference/algorithms/centrality.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ Centrality

.. autofunction:: raphtory.algorithms.hits

.. autofunction:: raphtory.algorithms.betweenness_centrality
1 change: 1 addition & 0 deletions examples/rust/src/bin/hulongbay/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ fn try_main() -> Result<(), Box<dyn Error>> {
let components = weakly_connected_components(&graph, 5, Some(16));

components
.result
.into_iter()
.counts_by(|(_, cc)| cc)
.iter()
Expand Down
14 changes: 8 additions & 6 deletions examples/rust/src/bin/lotr/main.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
use itertools::Itertools;
use raphtory::{
algorithms::pathing::temporal_reachability::temporally_reachable_nodes, core::utils::hashing,
graph_loader::source::csv_loader::CsvLoader, prelude::*,
Expand Down Expand Up @@ -104,9 +103,12 @@ fn main() {
assert!(graph.has_vertex(gandalf));
assert_eq!(graph.vertex(gandalf).unwrap().name(), "Gandalf");

let r = temporally_reachable_nodes(&graph, None, 20, 31930, vec!["Gandalf"], None);
assert_eq!(
r.result.keys().sorted().collect_vec(),
vec!["Gandalf", "Saruman", "Wormtongue"]
)
let r: Vec<String> = temporally_reachable_nodes(&graph, None, 20, 31930, vec!["Gandalf"], None)
.get_all_values()
.into_iter()
.flatten()
.map(|(_, s)| s)
.collect();

assert_eq!(r, vec!["Gandalf", "Saruman", "Wormtongue"])
}
2 changes: 2 additions & 0 deletions python/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ fn raphtory(py: Python<'_>, m: &PyModule) -> PyResult<()> {
algorithm_module,
dijkstra_single_source_shortest_paths,
global_reciprocity,
betweenness_centrality,
all_local_reciprocity,
triplet_count,
local_triangle_count,
Expand Down Expand Up @@ -107,6 +108,7 @@ fn raphtory(py: Python<'_>, m: &PyModule) -> PyResult<()> {
neo4j_movie_graph,
stable_coin_graph,
reddit_hyperlink_graph,
karate_club_graph,
);
m.add_submodule(graph_loader_module)?;

Expand Down
46 changes: 36 additions & 10 deletions python/tests/test_algorithms.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,16 @@ def test_degree_centrality():
from raphtory.algorithms import degree_centrality

g = Graph()
g.add_edge(0, 0, 1, {})
g.add_edge(0, 0, 2, {})
g.add_edge(0, 0, 3, {})
g.add_edge(0, 1, 2, {})
g.add_edge(0, 1, 3, {})
assert degree_centrality(g).get_all() == {
"0": 1.0,
g.add_edge(0, 1, 4, {})
g.add_edge(0, 2, 3, {})
g.add_edge(0, 2, 4, {})
assert degree_centrality(g).get_with_names() == {
"1": 1.0,
"2": 2 / 3,
"2": 1.0,
"3": 2 / 3,
"4": 2 / 3,
}


Expand Down Expand Up @@ -45,12 +45,12 @@ def test_single_source_shortest_path():
g.add_edge(0, 2, 4, {})
res_one = single_source_shortest_path(g, 1, 1)
res_two = single_source_shortest_path(g, 1, 2)
assert res_one.get_all() == {"1": ["1"], "2": ["1", "2"], "4": ["1", "4"]}
assert res_one.get_with_names() == {"1": ["1"], "2": ["1", "2"], "3": None, "4": ["1", "4"]}
assert (
res_two.get_all()
res_two.get_with_names()
== {"1": ["1"], "2": ["1", "2"], "3": ["1", "2", "3"], "4": ["1", "4"]}
) or (
res_two.get_all()
res_two.get_with_names()
== {"1": ["1"], "3": ["1", "4", "3"], "2": ["1", "2"], "4": ["1", "4"]}
)

Expand Down Expand Up @@ -84,4 +84,30 @@ def test_dijsktra_shortest_paths():
dijkstra_single_source_shortest_paths(g, "A", ["F"], weight="NO")
assert "Weight property not found on edges" in str(excinfo.value)



def test_betweenness_centrality():
from raphtory import Graph
from raphtory.algorithms import betweenness_centrality
g = Graph()
edges = [
(0, 1),
(0, 2),
(0, 3),
(1, 2),
(1, 3),
(1, 4),
(2, 3),
(2, 4),
(2, 5),
(3, 2),
(3, 1),
(3, 3)
]
for e in edges:
g.add_edge(0, e[0], e[1], {})

res = betweenness_centrality(g, normalized=False)
assert res.get_with_names() == { "0": 0.0, '1': 1.0, "2": 4.0, "3": 1.0, "4": 0.0, "5": 0.0 }

res = betweenness_centrality(g, normalized=True)
assert res.get_with_names() == { "0": 0.0, '1': 0.05, "2": 0.2, "3": 0.05, "4": 0.0, "5": 0.0}
6 changes: 6 additions & 0 deletions python/tests/test_graph_gen.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
def test_karate_club():
from raphtory.graph_loader import karate_club_graph
g = karate_club_graph()
assert g.count_vertices() == 34
assert g.count_edges() == 155

29 changes: 15 additions & 14 deletions python/tests/test_graphdb.py
Original file line number Diff line number Diff line change
Expand Up @@ -1254,22 +1254,22 @@ def test_connected_components():
g = gen_graph()
actual = algorithms.weakly_connected_components(g, 20)
expected = {"1": 1, "2": 1, "3": 1, "4": 1, "5": 1, "6": 1, "7": 1, "8": 1}
assert actual.get_all() == expected
assert actual.get_with_names() == expected
assert actual.get("1") == 1


def test_empty_algo():
g = Graph()
assert algorithms.weakly_connected_components(g, 20).get_all() == {}
assert algorithms.pagerank(g, 20).get_all() == {}
assert algorithms.weakly_connected_components(g, 20).get_with_names() == {}
assert algorithms.pagerank(g, 20).get_with_names() == {}


def test_algo_result():
g = gen_graph()

actual = algorithms.weakly_connected_components(g, 20)
expected = {"1": 1, "2": 1, "3": 1, "4": 1, "5": 1, "6": 1, "7": 1, "8": 1}
assert actual.get_all() == expected
assert actual.get_with_names() == expected
assert actual.get("1") == 1
assert actual.get("not a node") == None
expected_array = [
Expand All @@ -1289,13 +1289,14 @@ def test_algo_result():
assert len(actual.group_by()[1]) == 8
assert type(actual.to_df()) == pandas.core.frame.DataFrame
df = actual.to_df()
expected_result = pd.DataFrame({"Key": ["1"], "Value": [1]})
row_with_one = df[df["Key"] == "1"]
expected_result = pd.DataFrame({"Key": [1], "Value": [1]})
row_with_one = df[df["Key"] == 1]
row_with_one.reset_index(inplace=True, drop=True)
print(row_with_one)
assert row_with_one.equals(expected_result)
# Algo Str u64
actual = algorithms.weakly_connected_components(g)
all_res = actual.get_all()
all_res = actual.get_with_names()
sorted_res = {k: all_res[k] for k in sorted(all_res)}
assert sorted_res == {
"1": 1,
Expand All @@ -1319,12 +1320,12 @@ def test_algo_result():
"7": 0.14074777909144864,
"8": 0.11786468661230831,
}
assert actual.get_all() == expected_result
assert actual.get_with_names() == expected_result
assert actual.get("Not a node") == None
assert len(actual.to_df()) == 8
# algo str vector
actual = algorithms.temporally_reachable_nodes(g, 20, 11, [1, 2], [4, 5])
assert sorted(actual.get_all()) == ["1", "2", "3", "4", "5", "6", "7", "8"]
assert sorted(actual.get_with_names()) == ["1", "2", "3", "4", "5", "6", "7", "8"]


def test_page_rank():
Expand All @@ -1340,7 +1341,7 @@ def test_page_rank():
"7": 0.14074777909144864,
"8": 0.11786468661230831,
}
assert actual.get_all() == expected
assert actual.get_with_names() == expected


def test_temporal_reachability():
Expand All @@ -1358,7 +1359,7 @@ def test_temporal_reachability():
"8": [],
}

assert actual.get_all() == expected
assert actual.get_with_names() == expected


# def test_generic_taint_loader():
Expand Down Expand Up @@ -1747,13 +1748,13 @@ def test_balance_algorithm():
]
for src, dst, val, time in edges_str:
g.add_edge(time, src, dst, {"value_dec": val})
result = algorithms.balance(g, "value_dec", PyDirection("BOTH"), None).get_all()
Haaroon marked this conversation as resolved.
Show resolved Hide resolved
result = algorithms.balance(g, "value_dec", PyDirection("BOTH"), None).get_with_names()
assert result == {"1": -26.0, "2": 7.0, "3": 12.0, "4": 5.0, "5": 2.0}

result = algorithms.balance(g, "value_dec", PyDirection("IN"), None).get_all()
result = algorithms.balance(g, "value_dec", PyDirection("IN"), None).get_with_names()
assert result == {"1": 6.0, "2": 12.0, "3": 15.0, "4": 20.0, "5": 2.0}

result = algorithms.balance(g, "value_dec", PyDirection("OUT"), None).get_all()
result = algorithms.balance(g, "value_dec", PyDirection("OUT"), None).get_with_names()
assert result == {"1": -32.0, "2": -5.0, "3": -3.0, "4": -15.0, "5": 0.0}


Expand Down
10 changes: 10 additions & 0 deletions raphtory-graphql/src/model/algorithm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,15 @@ impl From<(String, f64)> for Pagerank {
}
}

impl From<(String, Option<f64>)> for Pagerank {
fn from((name, rank): (String, Option<f64>)) -> Self {
Self {
name,
rank: rank.unwrap_or_default(), // use 0.0 if rank is None
}
}
}

impl From<(String, OrderedFloat<f64>)> for Pagerank {
fn from((name, rank): (String, OrderedFloat<f64>)) -> Self {
let rank = rank.into_inner();
Expand Down Expand Up @@ -134,6 +143,7 @@ impl Algorithm for Pagerank {
let tol = ctx.args.get("tol").map(|v| v.f64()).transpose()?;
let binding = unweighted_page_rank(graph, iter_count, threads, tol, true);
let result = binding
.get_with_names()
.into_iter()
.map(|pair| FieldValue::owned_any(Pagerank::from(pair)));
Ok(Some(FieldValue::list(result)))
Expand Down
Loading
Loading