Skip to content

Commit

Permalink
added real_timestamp_of_id, default_config, and ability
Browse files Browse the repository at this point in the history
added Helper.real_timestamp_of_id
added default configuration so app doesn’t crash if user didn’t set
config
added Erlang Node name as an option for nodes
updated documentation, version, and change_log
  • Loading branch information
weixiyen committed Feb 9, 2017
1 parent 8623e80 commit 70d5e90
Show file tree
Hide file tree
Showing 5 changed files with 45 additions and 14 deletions.
7 changes: 7 additions & 0 deletions changelog.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Changelog

## v 0.0.2

- added Helper.real_timestamp_of_id
- added default configuration so app doesn’t crash if user didn’t set config
- added Erlang Node name as an option for nodes
- updated documentation

## v 0.0.1

Initial Release
10 changes: 9 additions & 1 deletion lib/snowflake/helper.ex
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,21 @@ defmodule Snowflake.Helper do
use Bitwise

@doc """
Get timestamp in ms from epoch from any snowflake ID
Get timestamp in ms from your config epoch from any snowflake ID
"""
@spec timestamp_of_id(integer) :: integer
def timestamp_of_id(id) do
id >>> 22
end

@doc """
Get timestamp from computer epoch - January 1, 1970, Midnight
"""
@spec real_timestamp_of_id(integer) :: integer
def real_timestamp_of_id(id) do
timestamp_of_id(id) + Snowflake.Utils.epoch()
end

@doc """
Get bucket value based on segments of N days
"""
Expand Down
10 changes: 7 additions & 3 deletions lib/snowflake/utils.ex
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,17 @@ defmodule Snowflake.Utils do
Utility functions intended for Snowflake application.
epoch() and machine_id() are useful for inspecting in production.
"""
@default_config [
nodes: [],
epoch: 0
]

@doc """
Grabs epoch from config value
"""
@spec epoch() :: integer
def epoch() do
Application.get_env(:snowflake, :epoch)
Application.get_env(:snowflake, :epoch) || @default_config[:epoch]
end

@doc """
Expand All @@ -18,8 +22,8 @@ defmodule Snowflake.Utils do
"""
@spec machine_id() :: integer
def machine_id() do
nodes = Application.get_env(:snowflake, :nodes)
host_addrs = [hostname(), fqdn()] ++ ip_addrs()
nodes = Application.get_env(:snowflake, :nodes) || @default_config[:nodes]
host_addrs = [hostname(), fqdn(), Node.self()] ++ ip_addrs()

case MapSet.intersection(MapSet.new(host_addrs), MapSet.new(nodes)) |> Enum.take(1) do
[matching_node] -> Enum.find_index(nodes, fn node -> node == matching_node end)
Expand Down
2 changes: 1 addition & 1 deletion mix.exs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
defmodule Snowflake.Mixfile do
use Mix.Project

@version "0.0.1"
@version "0.0.2"
@url "https://github.com/blitzstudios/snowflake"
@maintainers ["Weixi Yen"]

Expand Down
30 changes: 21 additions & 9 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,15 @@ def deps do
end
```

Specify the nodes in your config. If your running a cluster, specify all the nodes in the cluster that snowflake runs on.
```elixir
def application do
[applications: [:snowflake]]
end
```

Specify the nodes in your config. If you're running a cluster, specify all the nodes in the cluster that snowflake runs on.

- **nodes** can be Public IPs, Private IPs, Hostnames, or FQDNs
- **nodes** can be Erlang Node Names, Public IPs, Private IPs, Hostnames, or FQDNs
- **epoch** should not be changed once you begin generating IDs and want to maintain sorting
- There should be no more than 1 snowflake generator per node, or you risk duplicate potential duplicate snowflakes on the same node.

Expand All @@ -28,7 +34,7 @@ Generating an ID is simple.

```elixir
Snowflake.next_id()
# => 828746161129414660
# => {:ok, 54974240033603584}
```

## Helper functions
Expand All @@ -46,27 +52,33 @@ Snowflake.Helper.bucket(30, :days)
Or if we want to know which bucket a snowflake ID should belong to, given we are
bucketing by every 30 days.
```elixir
Snowflake.Helper.bucket(30, :days, 828746161129414661)
# => 76
Snowflake.Helper.bucket(30, :days, 54974240033603584)
# => 5
```

Or if we want to know how many ms elapsed from epoch
```elixir
Snowflake.Helper.timestamp_of_id(828746161129414661)
Snowflake.Helper.timestamp_of_id(54974240033603584)
# => 197588482172
```

Or if we want to know how many ms elapsed from computer epoch (January 1, 1970 midnight). We can use this to derive an actual calendar date.
```elixir
Snowflake.Helper.real_timestamp_of_id(54974240033603584)
# => 1486669389497
```

## NTP

Keep your nodes in sync with [ntpd](https://en.wikipedia.org/wiki/Ntpd) or use
your VM equivalent as snowflake depends on OS time. ntpd's job is to slow down
or speed up the clock so that it syncs the time with the network time.
or speed up the clock so that it syncs os time with your network time.

## Architecture

Snowflake allows the user to specify the nodes in the cluster, each representing a machine. Snowflake at startup inspects itself for IP and Host information and derives its machine_id from the location of itself in the list of nodes defined in the config.
Snowflake allows the user to specify the nodes in the cluster, each representing a machine. Snowflake at startup inspects itself for Node, IP and Host information and derives its machine_id from the location of itself in the list of nodes defined in the config.

Machine ID is defaulted to **1023** if there is no configuration or if snowflake does not find a match to remain highly available. It is important to specify the correct IPs / Hostnames / FQDNs for the nodes in a production environment to avoid any chance of snowflake collision.
Machine ID is defaulted to **1023** if snowflake is not able to find itself within the specified config. It is important to specify the correct IPs / Hostnames / FQDNs for the nodes in a production environment to avoid any chance of snowflake collision.

## Benchmarks

Expand Down

0 comments on commit 70d5e90

Please sign in to comment.