You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi ,
i used custom script in ARM Template while creating HDInsight Spark Cluster. The template was getting executed successfully , but the script was not getting executed. I used some environment variables in my script which was not getting resolved. I spent a lot of time to figure it out the issue.
"scriptActions": [{
"name": "createLivyFolder",
"uri": "[parameters('scriptActionUri')]"
}]
The below challenges i faced
Not able to get the path of the folder from where the script was executed in HDInsight Cluster.
Not able to get the path of error log or Stack-trace of script action.
After spending a lot of time i got the log from the Azure blob storage and was able to get the cause of the problem.
Can you please add to the doc how to debug the script action in case of failure or get the stack trace in case of success/failure.
Please let us know if we want to use some environment variables in script action like $SPARK_HOME,
How to do that. The below statement was failing in script actions.
sudo cp postgresql-9.4.1210.jre7.jar $SPARK_HOME/jars/
The text was updated successfully, but these errors were encountered:
Hi ,
i used custom script in ARM Template while creating HDInsight Spark Cluster. The template was getting executed successfully , but the script was not getting executed. I used some environment variables in my script which was not getting resolved. I spent a lot of time to figure it out the issue.
"scriptActions": [{
"name": "createLivyFolder",
"uri": "[parameters('scriptActionUri')]"
}]
The below challenges i faced
After spending a lot of time i got the log from the Azure blob storage and was able to get the cause of the problem.
Can you please add to the doc how to debug the script action in case of failure or get the stack trace in case of success/failure.
Please let us know if we want to use some environment variables in script action like $SPARK_HOME,
How to do that. The below statement was failing in script actions.
sudo cp postgresql-9.4.1210.jre7.jar $SPARK_HOME/jars/
The text was updated successfully, but these errors were encountered: