Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

run parameters prototype #1121

Open
wants to merge 61 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
38fb15a
wip
0xbase12 Jul 12, 2017
5c349ea
wip
0xbase12 Jul 12, 2017
90be0aa
wip
0xbase12 Jul 12, 2017
0b71e24
Merge branch 'master' into issue#1112-run-paramtetes-prototype
0xbase12 Jul 12, 2017
30e8445
working sse prototype
0xbase12 Jul 13, 2017
ae09522
sse and zookeeper working
0xbase12 Jul 14, 2017
f42e004
Merge branch 'master' into issue#1112-run-paramtetes-prototype
0xbase12 Jul 18, 2017
c816f8a
run with action prototype working and some unit test
0xbase12 Jul 18, 2017
6cb9db0
wip
0xbase12 Jul 19, 2017
8f0ed09
wip run parameters working with zookeeper for creation
0xbase12 Jul 20, 2017
28ae980
wip run parameter retrieve value from zookeeper
0xbase12 Jul 20, 2017
5a447b0
wip issue with sse
0xbase12 Jul 21, 2017
56d2acb
fix issue with nginx buffering and accept header logic implemented
0xbase12 Jul 22, 2017
e6c5280
run param edit set value in zk and fix issue with get-version from zo…
0xbase12 Jul 23, 2017
622f147
implement query for run param with sse and json
0xbase12 Jul 24, 2017
cbf8d6d
add unit tests and dependencies
0xbase12 Jul 24, 2017
aa6e37e
simple lifecycle test for run with embded zookeeper server working
0xbase12 Jul 25, 2017
63a813d
Merge branch 'issue#1112-run-paramtetes-prototype' into issue#1111-ne…
0xbase12 Jul 25, 2017
cbfbc7e
minor enhancement and more unit test for start run action
0xbase12 Jul 25, 2017
69d7f19
Merge branch 'master' into issue#1111-new-run-ssclj-resource
0xbase12 Jul 27, 2017
22a4ed8
run parameter life cycle tests and run id should be passed at creatio…
0xbase12 Jul 27, 2017
d83a4bf
minor, vmstate should be set by the client
0xbase12 Jul 27, 2017
5866979
add acl for un parameters
0xbase12 Jul 28, 2017
199d006
rename run-id to run-href
0xbase12 Jul 28, 2017
dbc3d37
SSE data is now the full json of run parameter and little more other …
0xbase12 Jul 28, 2017
585052c
Merge branch 'master' into issue#1111-new-run-ssclj-resource
konstan Aug 28, 2017
39a5dee
names of es fixtures changed
konstan Aug 28, 2017
d3ac0d4
Merge branch 'master' into issue#1111-new-run-ssclj-resource
konstan Aug 28, 2017
65e23f6
Merge branch 'master' into issue#1111-new-run-ssclj-resource
konstan Aug 29, 2017
50916db
add values to run parameter query
0xbase12 Sep 1, 2017
023cc36
add type to run parameter spec
0xbase12 Sep 1, 2017
52075aa
store values in elasticsearch
0xbase12 Sep 1, 2017
65baf4c
Merge branch 'master' into issue#1111-new-run-ssclj-resource
0xbase12 Sep 1, 2017
c12bd33
Merge branch 'master' into issue#1111-new-run-ssclj-resource
0xbase12 Sep 4, 2017
39b7d6b
Merge branch 'issue#1111-new-run-ssclj-resource' of ssh://github.com/…
0xbase12 Sep 4, 2017
266984b
renaming run resource to deployment
0xbase12 Sep 5, 2017
a63d623
update deployment attribute state when run parameter is updated
0xbase12 Sep 5, 2017
591feab
in lifecycle test clean all zk nodes after each test
0xbase12 Sep 6, 2017
afb1b85
FIXME: temporarily adding explicit dependency on clojure/tools.reader
konstan Sep 6, 2017
54a7d0d
Merge branch 'issue#1111-new-run-ssclj-resource' of github.com:slipst…
konstan Sep 6, 2017
d953909
removed unused requires
konstan Sep 6, 2017
cf9faf6
WIP update of state-complete for node instance have special behavior …
0xbase12 Sep 6, 2017
bbf5162
WIP for lock and deployment state move
0xbase12 Sep 8, 2017
f0c8ffd
next-state working and idempotent
0xbase12 Sep 11, 2017
5d509e5
SSE should answer in json
0xbase12 Sep 11, 2017
f7d363f
enhance http accept header interpretation
0xbase12 Sep 11, 2017
2984d4f
enhance code and fix deployment-href in deployment parameter
0xbase12 Sep 11, 2017
24dfaf1
rename deployment-href to deployment in deployment parameter and minor
0xbase12 Sep 11, 2017
870325b
Merge branch 'master' into issue#1111-new-run-ssclj-resource
0xbase12 Sep 12, 2017
86a1d2a
abort deployment possible and creation of deployment from java code
0xbase12 Sep 14, 2017
bead98d
fix deployment spec, add abort
0xbase12 Sep 14, 2017
54d863c
new header auth for java, ssclj deployment template
0xbase12 Sep 15, 2017
d923418
fix authentification regression
0xbase12 Sep 18, 2017
ce5973e
Merge branch 'master' into issue#1111-new-run-ssclj-resource
konstan Sep 18, 2017
0578af4
add conversion from java run to deployment and fake http server for t…
0xbase12 Sep 18, 2017
af9e526
Merge branch 'master' into issue#1111-new-run-ssclj-resource
0xbase12 Sep 18, 2017
d3d881a
Merge branch 'issue#1111-new-run-ssclj-resource' of ssh://github.com/…
0xbase12 Sep 18, 2017
41a1327
fix lifecycle tests for deployment
0xbase12 Sep 19, 2017
a683faf
set java runtime parameter are propagated to ssclj and dyn creation o…
0xbase12 Sep 19, 2017
daaa39f
terminate action on deployment to be continued
0xbase12 Sep 20, 2017
95a3763
minor
0xbase12 Sep 21, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ public void terminate(Run run, User user) throws SlipStreamException {

validateCredentials(user);

List<String> instanceIds = getCloudNodeInstanceIds(run);
List<String> instanceIds = getCloudNodeInstanceIds(run); //TODO KB extract ids for each cloud service for terminate
if(instanceIds.isEmpty()){
throw new SlipStreamClientException("There is no instances to terminate");
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
import com.sixsq.slipstream.persistence.Run;
import com.sixsq.slipstream.persistence.User;
import com.sixsq.slipstream.persistence.UserParameter;
import com.sixsq.slipstream.statemachine.StateMachine;
//import com.sixsq.slipstream.statemachine.StateMachine;
import com.sixsq.slipstream.statemachine.States;

public class Terminator {
Expand All @@ -44,7 +44,7 @@ public static int purge() throws ConfigurationException, ValidationException {
u = User.loadByName(u.getName());
int timeout = u.getTimeout();

List<Run> old = Run.listOldTransient(u, timeout);
List<Run> old = Run.listOldTransient(u, timeout); //TODO kb search for timeout on ssclj deployment
for (Run r : old) {
EntityManager em = PersistenceUtil.createEntityManager();
try {
Expand All @@ -68,7 +68,7 @@ public static int purge() throws ConfigurationException, ValidationException {
public static void purgeRun(Run run) throws SlipStreamException {
Run.abort("The run has timed out", run.getUuid());

boolean isGarbageCollected = Run.isGarbageCollected(run);
boolean isGarbageCollected = Run.isGarbageCollected(run); //TODO kb Garbage collected run
Run.setGarbageCollected(run);
run = run.store();

Expand All @@ -94,18 +94,20 @@ public static void terminate(String runResourceUri) throws SlipStreamException {
Run run = Run.load(runResourceUri, em);
User user = User.loadByName(run.getUser());

StateMachine sc = StateMachine.createStateMachine(run);
//StateMachine sc = StateMachine.createStateMachine(run);

if (sc.canCancel()) {
sc.tryAdvanceToCancelled();
terminateInstances(run, user);
} else {
if (sc.getState() == States.Ready) {
sc.tryAdvanceToFinalizing();
}
terminateInstances(run, user);
sc.tryAdvanceState(true);
}
//if (sc.canCancel()) {
// sc.tryAdvanceToCancelled();
// terminateInstances(run, user);
//} else {
// if (sc.getState() == States.Ready) {
// sc.tryAdvanceToFinalizing();
// }
// terminateInstances(run, user);
// sc.tryAdvanceState(true);
//}

terminateInstances(run, user);

em.close();
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
package com.sixsq.slipstream.persistence;

import com.google.gson.annotations.SerializedName;
import com.sixsq.slipstream.acl.TypePrincipal;
import com.sixsq.slipstream.acl.TypePrincipalRight;
import com.sixsq.slipstream.util.SscljProxy;
import com.sixsq.slipstream.acl.ACL;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;

public class Deployment {
private static final String DEPLOYMENT_RESOURCE = "api/deployment";

@SuppressWarnings("unused")
private String id;

@SuppressWarnings("unused")
@SerializedName("module-resource-uri")
private String moduleResourceUri;

@SuppressWarnings("unused")
private String type;

@SuppressWarnings("unused")
private String category;

@SuppressWarnings("unused")
private boolean mutable;

@SuppressWarnings("unused")
@SerializedName("keep-running")
private boolean keepRunning;

@SuppressWarnings("unused")
private ArrayList<String> tags;

@SuppressWarnings("unused")
private ACL acl;

@SuppressWarnings("unused")
private String state;

@SuppressWarnings("unused")
private HashMap<String, Node> nodes = new HashMap<>();

public void setId(String id) {
this.id = id;
}

public void setModuleResourceUri(String moduleResourceUri) {
this.moduleResourceUri = moduleResourceUri;
}

public void setType(String type) {
this.type = type;
}

public void setCategory(String category) {
this.category = category;
}

public void setMutable(boolean mutable) {
this.mutable = mutable;
}

public void setAcl(ACL acl) {
this.acl = acl;
}

public void setState(String state) {
this.state = state;
}

public void setKeepRunning(boolean keepRunning) {
this.keepRunning = keepRunning;
}

public void setNodes(HashMap<String, Node> nodes) {
this.nodes = nodes;
}

public void setTags(ArrayList<String> tags) {
this.tags = tags;
}

public String getId() {
return id;
}

public String getModuleResourceUri() {
return moduleResourceUri;
}

public String getType() {
return type;
}

public String getCategory() {
return category;
}

public boolean isMutable() {
return mutable;
}

public ACL getAcl() {
return acl;
}

public String getState() {
return state;
}

public boolean isKeepRunning() {
return keepRunning;
}

public ArrayList<String> getTags() {
return tags;
}

public HashMap<String, Node> getNodes() {
return nodes;
}

class Node {
class Parameter {
String description;
String value;

Parameter (String value, String description) {
this.value = value;
this.description = description;
}
}
@SerializedName("runtime-parameters")
HashMap<String, Parameter> runtimeParameters = new HashMap<>();

Node (String nodeName, Run run) {
for (RuntimeParameter rp : run.getRuntimeParameters().values()) {
if (rp.getGroup().equals(nodeName)) {
runtimeParameters.put(rp.getName(), new Parameter(rp.getValue(), rp.getDescription()));
}
}
}
}

public Deployment(Run run) {
this.id = "deployment/" + run.getUuid();
this.moduleResourceUri = run.getRefqname();
this.type = run.getType().name();
this.category = run.getCategory().name();
this.mutable = run.isMutable();
this.state = run.getState().name();

for (String nodeName: run.getNodeNamesList()) {
this.nodes.put(nodeName, new Node(nodeName, run));
}

try {
this.tags = new ArrayList<>(Arrays.asList(run.getRuntimeParameterValue("ss:tags").split(",")));
} catch (Exception ignored) { }

TypePrincipal owner = new TypePrincipal(TypePrincipal.PrincipalType.USER, run.getUser());
List<TypePrincipalRight> rules = new ArrayList<TypePrincipalRight>();
rules.add(new TypePrincipalRight(TypePrincipal.PrincipalType.USER, run.getUser(), TypePrincipalRight.Right.MODIFY));
this.acl = new ACL(owner, rules);
}

public String toJson() {
return SscljProxy.toJson(this);
}

public static void post(Deployment deployment) {
SscljProxy.post(DEPLOYMENT_RESOURCE, "internal ADMIN", deployment);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ public class Run extends Parameterized<Run, RunParameter> {
private static final Set<States> STOP_ACCOUNTING_STATES = new HashSet<States>(Arrays.asList(STOP_VALUES));


public static Run abortOrReset(String abortMessage, String nodename,
public static Run abortOrReset(String abortMessage, String nodename, //TODO kb
String uuid) {
EntityManager em = PersistenceUtil.createEntityManager();
EntityTransaction transaction = em.getTransaction();
Expand All @@ -142,7 +142,7 @@ public static Run abortOrReset(String abortMessage, String nodename,
return run;
}

public static Run abortOrReset(String abortMessage, String nodename,
public static Run abortOrReset(String abortMessage, String nodename, //TODO kb
EntityManager em, String uuid) {

Run run = Run.loadFromUuid(uuid, em);
Expand Down Expand Up @@ -177,7 +177,7 @@ public static Run abortOrReset(String abortMessage, String nodename,
return run;
}

private static void setGlobalAbortState(String abortMessage,
private static void setGlobalAbortState(String abortMessage, //TODO kb
RuntimeParameter globalAbort) {
globalAbort.setValue(abortMessage);
globalAbort.store();
Expand All @@ -192,7 +192,7 @@ public static Run abort(String abortMessage, String uuid) {
if (!globalAbort.isSet()) {
setGlobalAbortState(abortMessage, globalAbort);
}
if (run.state == States.Provisioning) {
if (run.state == States.Provisioning) { //TODO kb recovery mode
setRecoveryMode(run);
}
em.close();
Expand All @@ -210,7 +210,7 @@ private static String getNodeAbortKey(String nodeName) {
+ RuntimeParameter.ABORT_KEY;
}

private static RuntimeParameter getRecoveryModeParameter(Run run) {
private static RuntimeParameter getRecoveryModeParameter(Run run) { //TODO kb
return run.getRuntimeParameters().get(
RuntimeParameter.GLOBAL_RECOVERY_MODE_KEY);
}
Expand Down Expand Up @@ -331,7 +331,7 @@ private static List<RunView> convertRunsToRunViews(List<Run> runs, Map<String, L
return views;
}

private static RunView convertRunToRunView(Run run, Map<String, Long> vmCountPerRun) {
private static RunView convertRunToRunView(Run run, Map<String, Long> vmCountPerRun) { //TODO kb UI

if (run == null) {
return null;
Expand Down Expand Up @@ -480,7 +480,7 @@ public static List<Run> listOldTransient(User user) throws ConfigurationExceptio
}

@SuppressWarnings("unchecked")
public static List<Run> listOldTransient(User user, int timeout) throws ConfigurationException,
public static List<Run> listOldTransient(User user, int timeout) throws ConfigurationException, //TODO kb search for old transient deployment on ssclj
ValidationException {
if (timeout <= 0) {
timeout = DEFAULT_TIMEOUT;
Expand Down Expand Up @@ -1135,7 +1135,7 @@ public void addGroup(String group, String serviceName) {
@Attribute
@Column(length = 1024)
public String getGroups() {
getRuntimeParameters().get(RuntimeParameter.GLOBAL_NODE_GROUPS_KEY).setValue(groups);
getRuntimeParameters().get(RuntimeParameter.GLOBAL_NODE_GROUPS_KEY).setInitValue(groups);
return groups;
}

Expand Down
Loading