Skip to content

Configuration

Jump to bottom Edit New page
dkapoor edited this page Aug 20, 2014 · 38 revisions

Configuration topics:

Amount of Memory Karma Can Use

If you want to use Karma with files containing thousands of rows or ontologies containing hundreds of classes and properties, then you need to increase the amount of memory allocated to Karma.

  • If using Jetty, define the MAVEN_OPTS environment variable and set it to a value larger than Xmx1024m

If you are using bash (e.g., on the Mac), put the following in the .profile file found in your home directory. The example shows how to give Karma 4GB of memory. BTW, you can also put in your .profile a convenient alias to invoke Karma by typing karma :

export MAVEN_OPTS=-Xmx4096m
alias karma="cd ~/Web-Karma/karma-web;mvn jetty:run"
  • If using tomcat define the CATALINA_OPTS environment variable and set it to a value larger than Xmx1024m

If you are using bash (e.g., on the Mac), put the following in the .profile file found in your home directory. The example shows how to give Karma 4GB of memory. BTW, you can also put in your .profile a convenient alias to invoke Karma by typing karma :

export CATALINA_OPTS="$CATALINA_OPTS -Xms2048m" 
export CATALINA_OPTS="$CATALINA_OPTS -Xmx4096m"
alias karma="cd ~/Web-Karma/karma-web;mvn jetty:run"

Karma User Home

Karma stores all user settings in folder {user.home}/karma. You can change this default location by changing it from the footer of Karma UI. Let this be called as the KARMA_USER_HOME The {user.home}/karma folder contains the following subfolders:

  • AlignmentGraph: you can ignore this folder, it is for a new feature we have not released yet.
  • SemanticTypeModels: stores the learned semantic types.
  • JSON: you can ignore this folder. It is for a new save as JSON feature.
  • UserPrefs: file to store your user preferences.
  • UserUploadedFiles: a copy of all the files you loaded in Karma from your local disk. You can delete them if you want.
  • history: files for internal Karma use.
  • preloaded-ontologies: put here the owl and rdf files you want Karma to load each time you load the Karma Web page.
  • python: Allows you to define your own Python library.

All published data (models, rdf) is under karma-web/src/main/webapp/publish folder

Information Karma Learns Semantic Types

Karma includes a component that learns to assign semantic types to columns of data. Each time you assign a semantic type, Karma stores information about the semantic type in in the karma-web/src/main/webapp/SemanticTypeModels subdirectory of your Karma home directory.

Your SemanticTypeModelsdirectory can grow large and it's OK to remove it if it gets over a megabyte. If you do, Karma will stop offering you suggestions when you assign semantic types. Karma will learn from subsequent assignments and start offering suggestions again.

Automatically Loading Commonly-Used Ontologies preloaded-ontologies

When Karma starts, it automatically loads all the ontology files found in the preloaded-ontologies subdirectory of the Karma user-home directory. By default the karma user-home directory will be <user.home>/karma.

Here is an example of a preloaded-ontlogies folder with 6 ontology files that Karma loads each time it starts, i.e., each time you load the Karma web page.

szeke:~ szekely> cd karma
szeke:karma szekely> ls preloaded-ontologies
dcterms.rdf	dublincore.owl	foaf.rdf	skos-xl.rdf	skos.rdf	vp-basic.owl

Modeling Configuration

Karma modeling behavior can be configured using the KARMA_USER_HOME/Config/modeling.properties file. Here is a sample modeling.properties file:

##########################################################################################
#
# Graph Builder
#
##########################################################################################
manual.alignment=false

node.closure=true

properties.direct=true
properties.indirect=true
properties.subclass=true
properties.with.only.domain=true
properties.with.only.range=true
properties.without.domain.range=false

##########################################################################################
#
# Prefixes
#
##########################################################################################
karma.source.prefix=http://isi.edu/integration/karma/sources/
karma.service.prefix=http://isi.edu/integration/karma/services/

##########################################################################################
#
# Model Learner
#
##########################################################################################
learner.enabled=true

max.queued.mappings=100
max.candidate.models=5
multiple.same.property.per.node=true

# scoring coefficients, should be in range [0..1]
scoring.confidence.coefficient=1.0
scoring.coherence.coefficient=1.0
scoring.size.coefficient=1.0

models.json.dir=JSON/
models.graphviz.dir=GRAPHVIZ/
alignment.graph.dir=AlignmentGraph/
##########################################################################################
#
# Other Settings
#
##########################################################################################
models.display.nomatching=false
history.store.old=false

UI Configuration

Karma UI can be configured using the KARMA_USER_HOME/Config/ui.properties file. Here is a sample ui.properties file:

google.earth.enabled=true
max.loaded.classes=-1
max.loaded.properties=-1
d3.display.charts=true
  • To disable google earth, set google.earth.enabled=true
  • The Karma UI by default loads all classes and properties as a list. This can slow the system if the size of the Ontologies is large. In those cases, you can set max.loaded.classes to a small number, example: max.loaded.classes=100 so that if the number of classes increases beyond 100, it would not render the class list, but user will have to manually type in the class name. Karma does provide typeahead to assist the user as he enters the class name. The default value is max.loaded.classes=-1 and will cause the class list to always show.
  • Similar to classes, you can also limit the properties list using max.loaded.properties
  • If the source is very large, generate the charts for the source can take a lot of time, leading to a very slow loading of the worksheet. To make it faster, you can disable the loading of the charts by setting d3.display.charts=false

User Preferences UserPrefs

User preferences enable you to control how much information Karma shows on the screen. To specify your preferences you need to enter them in a JSON file in the <user.home>/karma/UserPrefs directory. Here is an example of preferences stored in WSP1.json:

szeke:UserPrefs szekely> pwd
/Users/szekely/karma/UserPrefs
szeke:UserPrefs szekely> cat WSP1.json 
{
    "ViewPreferences": {
        "defaultRowsToShowInTopTables": 150,
        "maxCharactersInCell": 80,
        "maxCharactersInHeader": 10,
        "maxRowsToShowInNestedTables": 25
    }
}

  • defaultRowsToShowInTopTables : the number of top-level rows shown in the browser. The worksheet may have many more rows, and in the current version you cannot scroll to rows that are not visible (the little controls on the screen don't work).

  • maxCharactersInCell : maximum number of characters of a cell value that will be shown on the screen. To see the full value click on the Expand menu.

  • maxCharactersInHeader : maximum number of characters of a column header that will be shown on the screen. Currently, there is no way in the user interface to see the full string.

  • maxRowsToShowInNestedTables : Karma supports worksheets where the values of cells can themselves be tables. These are called nested tables. This parameter controls the number of rows from nested that will be shown in the browser.

Karma keeps track of different users by defining a workspace for each one: this is why the preference files are called WSP1.json, WSP2.json, etc. When you install Karma in your local machine you will most likely only have one preferences file so simple edit the one you have. Most likely it is called WSP1.json. Note: a future version of Karma will enable users to login and workspaces will be tied to login names.

Your preferences file may also contain information that Karma saves so that it can use it in future sessions. Examples include the URLs for database connections and default namespaces for RDF export. You can edit the information in these commands so that Karma uses the values you enter in the future. If you subsequently change the settings in the interface, those settings will be stored in your user preferences file, overwriting your edits.

The UserPrefs directory also contains a WorkspacePref.template file. Karma copies the preferences for each workspace from this template file, so you can edit it to change the default preferences for all workspaces.

Files Where Karma Saves Your Work

This feature is in transition as of 27 April 2014, and the system is not working as documented here. Karma does save the command history here, but there is currently no command to load it again. We are transitioning the command history into the R2RML model file. We expect to have the auto-save feature working in May 2014.

Karma automatically saves a history file containing all the commands you perform in the user interface. This is why there is no Save button in Karma, it saves your work behind the scenes as you go. The history files are stored in the KARMA_USER_HOME/History directory. Here is an example of of a History directory:

szeke:History szekely> pwd
/Users/szekely/karma/History
szeke:History szekely> ls
WSP1_DMA American Dataset_DMA American Datasetcsv.json		WSP1_crystal-bridges-records_Sheet1.json
WSP1_DMA-artist-labelscsv.json					WSP1_fis_departmentsdat.json
WSP1_DMA-artwork-labelscsv.json					WSP1_ima-artworks-demoxml.json
WSP1_alignment-geonames-saam-jarowinkler-01-07axml.json		WSP1_npg-artist-deathjson.json

The file names match the names of the files you load in Karma, and they contain all the state-changing commands that you performed on that file (e.g., publish commands are not in the history). You should not edit the content of these files, but you may want to make copies or rename them so that you can preserve the models or transformations you defined for a file. If you don't preserve copies of these files, Karma will overwrite them with whatever new commands you perform in the user interface.

Sometimes, things get messed up and Karma keeps crashing when you try to import a file. It may be that the history file got corrupted, so renaming it or deleting it may allow you to load a file that worked fine before.

Karma has a Reset command that can delete all the history files. Warning: this is a pretty drastic measure to take so before hitting the Reset button try to move some of the history files to other directories.

Locations of Files Karma Publishes

The Publish commands in Karma publish different kinds of files. Karma makes these files available as links in the Web browser, so they are not in your KARMA_USER_HOME folder. Instead, Karma puts these files in

Web-Karma/karma-web/src/main/webapp/publish/

27 April 2014: need to review this part of the documentation given that the new release moved some of these files to other places.

Karma publishes the following types of files:

  • RDF files containing worksheet data converted to RDF according to a model
  • Model containing an RDF representation of the model
  • Worksheet History containing a JSON representation of all the commands done on a worksheet
  • CSV representations of worksheet data
  • KML and Shapefile representations of worksheet data when data is modeled using a geospatial ontology

RDF files

Karma stores the RDF files generated from worksheets in the karma-web/src/main/webapp/RDF subdirectory of your Karma home directory. The files do not have particularly descriptive names, but Karma shows them as links in the user interface. If you lose track of the files, you can look for them in this directory. Here is an example:

szeke:RDF szekely> pwd
/Users/szekely/Web-Karma/karma-web/src/main/webapp/RDF
szeke:RDF szekely> ls -lt
total 848
-rw-r--r--  1 szekely  staff   78930 May 14 18:49 WSP1VW1.n3
-rw-r--r--  1 szekely  staff   46323 Apr 27 20:17 WSP1VW3.n3
-rw-r--r--  1 szekely  staff   29950 Apr 27 20:07 WSP1VW2.n3

Model Files

The model files contain the RDF representation of the models you build in Karma. You need these files if you want to use the batch RDF generation facility in Karma. in the karma-web/src/main/webapp/repository subdirectory of your Karma home directory.

Karma generates model files for sources and services and puts them in separate subdirectories. The model files for sources are stored in karma-web/src/main/webapp/repository/sources. Karma generates a new file each time you publish a model. You can determine from the name of the file which source it corresponds to. Here is an example:

szeke:sources szekely> pwd
/Users/szekely/Web-Karma/karma-web/src/main/webapp/repository/sources
szeke:sources szekely> ls
AAT_CONTRIB_RELS_NOTE_2C213AAE-8A33-5BEF-6CBF-ABFF497A483C.n3
DMA American Dataset_DMA American Dataset.csv_36DBBD9F-FDC8-AA9C-6AF5-16F8B43478F7.n3

Worksheet History Files

You can explicitly ask Karma to publish the worksheet history and display a link to it. The files are stored where all history files are stored. See Files Where Karma Saves Your Work.

CSV Files

You can use Karma to publish CSV files of your worksheet data. The CSV are stored in the karma-web/src/main/webapp/publish/CSV subdirectory of your Karma home directory. Here is an example:

szeke:CSV szekely> pwd
/Users/szekely/Web-Karma/karma-web/src/main/webapp/publish/CSV
szeke:CSV szekely> ls
AAT_SUBJECT.csv						alignment-geonames-saam-jarowinkler-01-07a.xml.csv
DMA American Dataset_DMA American Dataset.csv		all-met.json.csv

KML and Shapefiles

You can use Karma to publish KML and Shapefiles of your worksheet data that you model according to a geospatial ontology. These files are stored in the karma-web/src/main/webapp/publish/SpatialData subdirectory of your Karma home directory. Here is an example: Need an example with KML and Shapefiles. I have none.

szeke:SpatialData szekely> pwd
/Users/szekely/Web-Karma/karma-web/src/main/webapp/publish/SpatialData

Reset

The Karma Reset command is shown below. This command will delete the learned semantic types (more info: Files Where Karma Saves Your Work) and the histories (more info: Information Karma Learns).

Warning this command is not undoable and if you delete the histories you cannot get them back.

Add a custom footer

Home

Installation: One Click Install

Installation: Source Code

Configuration

Importing Data

Transforming Data

Modeling Data

Publishing Model

Working with Geospatial Data

Publishing Data

Batch Mode

Working with JSON-LD

Working with Karma using Map/Reduce

Karma As A Service

Auxiliary Services

Coding Advice and Standards

TroubleShooting

Working with GitHub

Quick Start Guide for Developers

Clone this wiki locally