Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arcbox 3.0 - Fixed app and ingress #2621

Merged
merged 47 commits into from
Jul 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
f83e02b
replace capi
zaidmohd May 29, 2024
25238a3
fix az login
zaidmohd May 29, 2024
4f05383
update to managed identity
zaidmohd May 30, 2024
1334251
enable system assigned identity
zaidmohd May 30, 2024
e67a636
remove debugging
zaidmohd May 30, 2024
af8d36b
comment resource provider, this will be part of pre-req
zaidmohd Jun 5, 2024
d5e59bb
update ip address
zaidmohd Jun 5, 2024
c6cba69
remove kv extension
zaidmohd Jun 5, 2024
f846010
add kubevip for service ip
zaidmohd Jun 5, 2024
541d526
fix kube config
zaidmohd Jun 6, 2024
905f555
fix nic name
zaidmohd Jun 6, 2024
246b7d3
fix nic name
zaidmohd Jun 6, 2024
2cfb09b
fix kube config
zaidmohd Jun 6, 2024
18cd0b2
Add role assignment dependency
zaidmohd Jun 6, 2024
8d81c2b
add kubeconfig
zaidmohd Jun 6, 2024
866e104
update https
zaidmohd Jun 6, 2024
e3e4d91
remove kube vip rbac url
zaidmohd Jun 6, 2024
0866319
remove spn details
zaidmohd Jun 7, 2024
6e832f8
Merge branch 'arcbox_3.0' of https://github.com/microsoft/azure_arc i…
zaidmohd Jun 7, 2024
7eb8d7c
update k3s scripts
zaidmohd Jun 7, 2024
0a0b5bb
k3s reset script changes
zaidmohd Jun 10, 2024
f368c43
Fix resource name
zaidmohd Jun 12, 2024
d00752e
update icon names
zaidmohd Jun 12, 2024
9841453
fix format
zaidmohd Jun 13, 2024
ac20698
fix format
zaidmohd Jun 13, 2024
26e47c0
fix inline kubectl format
zaidmohd Jun 13, 2024
7995479
add retry for k3s gitops
zaidmohd Jun 14, 2024
5cf9318
update github account
zaidmohd Jun 14, 2024
49d6fab
Merge branch 'arcbox_3.0' of https://github.com/microsoft/azure_arc i…
zaidmohd Jun 14, 2024
04d864f
Merge branch 'microsoft:arcbox_3.0' into arcbox_3.0
zaidmohd Jul 4, 2024
124622b
fix custom location issue
zaidmohd Jul 4, 2024
e356c03
fix install k3s and k3s nodes
zaidmohd Jul 4, 2024
443c8f4
fix k3s issue
zaidmohd Jul 5, 2024
81dcce7
Merge branch 'arcbox_3.0' of https://github.com/zaidmohd/azure_arc in…
zaidmohd Jul 5, 2024
a87ce06
Merge branch 'microsoft:arcbox_3.0' into arcbox_3.0
zaidmohd Jul 8, 2024
5239568
Merge branch 'microsoft:arcbox_3.0' into arcbox_3.0
zaidmohd Jul 11, 2024
a2f4bdd
update acr
zaidmohd Jul 11, 2024
7892907
Merge branch 'arcbox_3.0' of https://github.com/microsoft/azure_arc i…
zaidmohd Jul 16, 2024
3767a89
db-external-svc
zaidmohd Jul 18, 2024
3774223
fix app ingress
zaidmohd Jul 18, 2024
a9ac9d2
add namespace
zaidmohd Jul 19, 2024
a85d96b
Merge branch 'arcbox_3.0' of https://github.com/microsoft/azure_arc i…
zaidmohd Jul 22, 2024
a25aff4
remove ingress
zaidmohd Jul 22, 2024
60d262c
fix cname record
zaidmohd Jul 22, 2024
b667fca
cname
zaidmohd Jul 22, 2024
b29426d
fix cname issue
zaidmohd Jul 23, 2024
125fef3
fix app url
zaidmohd Jul 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 14 additions & 41 deletions azure_jumpstart_arcbox/artifacts/DataOpsAppDRScript.ps1
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
$Env:ArcBoxLogsDir = "C:\ArcBox\Logs"

$CName = "dataops"
$certdns = "$CName.jumpstart.local"
# $certdns = "$CName.jumpstart.local"
$appNamespace = "arc"
$sqlInstance = "aks-dr"

Expand All @@ -10,16 +10,6 @@ Start-Transcript -Path $Env:ArcBoxLogsDir\DataOpsAppDRScript.log
# Switch kubectl context to AKS DR
kubectx $sqlInstance

Write-Header "Adding CName Record for App"
$dcInfo = Get-ADDomainController
Do
{
$appIpaddress= kubectl get svc "dataops-ingress-nginx-ingress-controller" -o jsonpath="{.status.loadBalancer.ingress[0].ip}"
Start-Sleep -Seconds 5
} while ($null -eq $appIpaddress)
Add-DnsServerResourceRecord -ComputerName $dcInfo.HostName -ZoneName $dcInfo.Domain -A -Name "$CName-$sqlInstance" -AllowUpdateAny -IPv4Address $appIpaddress -TimeToLive 01:00:00 -AgeRecord
Add-DnsServerResourceRecordCName -Name $CName -ComputerName $dcInfo.HostName -HostNameAlias "$CName-$sqlInstance.jumpstart.local" -ZoneName jumpstart.local -TimeToLive 00:05:00

# Deploy the App and service
$appK3s = @"
apiVersion: apps/v1
Expand All @@ -40,7 +30,7 @@ spec:
spec:
containers:
- name: web
image: azurearcjumpstart.azurecr.io/demoapp:dr
image: jumpstartdev.azurecr.io/demoapp:dr
ports:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this our new ACR for images?

- containerPort: 80
volumeMounts:
Expand All @@ -59,7 +49,7 @@ metadata:
spec:
selector:
app: web
type: ClusterIP
type: LoadBalancer
ports:
- protocol: TCP
port: 80
Expand All @@ -69,34 +59,17 @@ spec:
Write-Header "Deploying App Resource"
$appK3s | kubectl apply -n $appNamespace -f -

# Deploy an Ingress Resource for the app
$appIngress = @"
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-tls
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
tls:
- hosts:
- "$certdns"
secretName: "$CName-secret"
rules:
- host: "$certdns"
http:
paths:
- pathType: ImplementationSpecific
backend:
service:
name: web-app-service
port:
number: 80
path: /
"@
Write-Header "Deploying App Ingress Resource"
$appIngress | kubectl apply -n $appNamespace -f -
# Write-Header "Adding CName Record for App"
$dcInfo = Get-ADDomainController
Do
{
Write-Host "Waiting for Web App Service, hold tight..."
$appIpaddress= kubectl get svc "web-app-service" -o jsonpath="{.status.loadBalancer.ingress[0].ip}"
Start-Sleep -Seconds 5
} while ($null -eq $appIpaddress)
Add-DnsServerResourceRecord -ComputerName $dcInfo.HostName -ZoneName $dcInfo.Domain -A -Name "$CName-$sqlInstance" -AllowUpdateAny -IPv4Address $appIpaddress -TimeToLive 01:00:00 -AgeRecord
Add-DnsServerResourceRecordCName -Name $CName -ComputerName $dcInfo.HostName -HostNameAlias "$CName-$sqlInstance.jumpstart.local" -ZoneName jumpstart.local -TimeToLive 00:05:00


Do {
Write-Host "Waiting for Web App pod, hold tight..."
Expand Down
104 changes: 38 additions & 66 deletions azure_jumpstart_arcbox/artifacts/DataOpsAppScript.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -3,54 +3,44 @@ $Env:ArcBoxLogsDir = "C:\ArcBox\Logs"
$Env:ArcBoxIconDir = "C:\ArcBox\Icons"

$CName = "jumpstartbooks"
$certdns = "$CName.jumpstart.local"
$password = "arcbox"
# $certdns = "$CName.jumpstart.local"
# $password = "arcbox"
$appNamespace = "arc"
$sqlInstance = "k3s"

Start-Transcript -Path $Env:ArcBoxLogsDir\DataOpsAppScript.log

# Add OpenSSL to path environment variable
$openSSL = "C:\Program Files\FireDaemon OpenSSL 3\bin"
$currentPathVariable = [Environment]::GetEnvironmentVariable("PATH", [EnvironmentVariableTarget]::Process)
$newPathVariable = $currentPathVariable + ";" + $openSSL
[Environment]::SetEnvironmentVariable("PATH", $newPathVariable, [EnvironmentVariableTarget]::Process)
# # Add OpenSSL to path environment variable
# $openSSL = "C:\Program Files\FireDaemon OpenSSL 3\bin"
# $currentPathVariable = [Environment]::GetEnvironmentVariable("PATH", [EnvironmentVariableTarget]::Process)
# $newPathVariable = $currentPathVariable + ";" + $openSSL
# [Environment]::SetEnvironmentVariable("PATH", $newPathVariable, [EnvironmentVariableTarget]::Process)

Write-Host "Generating a TLS Certificate"
$cert = New-SelfSignedCertificate -DnsName $certdns -KeyAlgorithm RSA -KeyLength 2048 -NotAfter (Get-Date).AddYears(1) -CertStoreLocation "Cert:\CurrentUser\My"
$certPassword = ConvertTo-SecureString -String $password -Force -AsPlainText
Export-PfxCertificate -Cert "cert:\CurrentUser\My\$($cert.Thumbprint)" -FilePath "$Env:TempDir\$CName.pfx" -Password $certPassword
Import-PfxCertificate -FilePath "$Env:TempDir\$CName.pfx" -CertStoreLocation Cert:\LocalMachine\Root -Password $certPassword
# Write-Host "Generating a TLS Certificate"
# $cert = New-SelfSignedCertificate -DnsName $certdns -KeyAlgorithm RSA -KeyLength 2048 -NotAfter (Get-Date).AddYears(1) -CertStoreLocation "Cert:\CurrentUser\My"
# $certPassword = ConvertTo-SecureString -String $password -Force -AsPlainText
# Export-PfxCertificate -Cert "cert:\CurrentUser\My\$($cert.Thumbprint)" -FilePath "$Env:TempDir\$CName.pfx" -Password $certPassword
# Import-PfxCertificate -FilePath "$Env:TempDir\$CName.pfx" -CertStoreLocation Cert:\LocalMachine\Root -Password $certPassword

openssl pkcs12 -in "$Env:TempDir\$CName.pfx" -nocerts -out "$Env:TempDir\$CName.key" -password pass:$password -passout pass:$password
openssl pkcs12 -in "$Env:TempDir\$CName.pfx" -clcerts -nokeys -out "$Env:TempDir\$CName.crt" -password pass:$password
openssl rsa -in "$Env:TempDir\$CName.key" -out "$Env:TempDir\$CName-dec.key" -passin pass:$password
# openssl pkcs12 -in "$Env:TempDir\$CName.pfx" -nocerts -out "$Env:TempDir\$CName.key" -password pass:$password -passout pass:$password
# openssl pkcs12 -in "$Env:TempDir\$CName.pfx" -clcerts -nokeys -out "$Env:TempDir\$CName.crt" -password pass:$password
# openssl rsa -in "$Env:TempDir\$CName.key" -out "$Env:TempDir\$CName-dec.key" -passin pass:$password

Write-Header "Creating Ingress Controller"
foreach ($cluster in @('k3s', 'aks-dr')) {
# Create K8s Ingress TLS secret
kubectx $cluster
kubectl -n $appNamespace create secret tls "$CName-secret" --key "$Env:TempDir\$CName-dec.key" --cert "$Env:TempDir\$CName.crt"
# Write-Header "Creating Ingress Controller"
# foreach ($cluster in @('k3s', 'aks-dr')) {
# # Create K8s Ingress TLS secret
# kubectx $cluster
# kubectl -n $appNamespace create secret tls "$CName-secret" --key "$Env:TempDir\$CName-dec.key" --cert "$Env:TempDir\$CName.crt"

# Deploy NGINX Ingress Controller
helm repo add nginx-stable https://helm.nginx.com/stable
helm repo update
helm install dataops-ingress nginx-stable/nginx-ingress
}
# # Deploy NGINX Ingress Controller
# helm repo add nginx-stable https://helm.nginx.com/stable
# helm repo update
# helm install dataops-ingress nginx-stable/nginx-ingress
# }

# Switch kubectl context to k3s
kubectx $sqlInstance

Write-Header "Adding CName Record for App"
$dcInfo = Get-ADDomainController
Do
{
$appIpaddress= kubectl get svc "dataops-ingress-nginx-ingress-controller" -o jsonpath="{.status.loadBalancer.ingress[0].ip}"
Start-Sleep -Seconds 5
} while ($null -eq $appIpaddress)
Add-DnsServerResourceRecord -ComputerName $dcInfo.HostName -ZoneName $dcInfo.Domain -A -Name "$CName-$sqlInstance" -AllowUpdateAny -IPv4Address $appIpaddress -TimeToLive 01:00:00 -AgeRecord
Add-DnsServerResourceRecordCName -Name $CName -ComputerName $dcInfo.HostName -HostNameAlias "$CName-$sqlInstance.jumpstart.local" -ZoneName jumpstart.local -TimeToLive 00:05:00

# Deploy the App and service
$appK3s = @"
apiVersion: apps/v1
Expand All @@ -71,7 +61,7 @@ spec:
spec:
containers:
- name: web
image: azurearcjumpstart.azurecr.io/demoapp
image: jumpstartdev.azurecr.io/demoapp
ports:
- containerPort: 80
volumeMounts:
Expand All @@ -90,7 +80,7 @@ metadata:
spec:
selector:
app: web
type: ClusterIP
type: LoadBalancer
ports:
- protocol: TCP
port: 80
Expand All @@ -100,34 +90,16 @@ spec:
Write-Header "Deploying App Resource"
$appK3s | kubectl apply -n $appNamespace -f -

# Deploy an Ingress Resource for the app
$appIngress = @"
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-tls
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
tls:
- hosts:
- "$certdns"
secretName: "$CName-secret"
rules:
- host: "$certdns"
http:
paths:
- pathType: ImplementationSpecific
backend:
service:
name: web-app-service
port:
number: 80
path: /
"@
Write-Header "Deploying App Ingress Resource"
$appIngress | kubectl apply -n $appNamespace -f -
Write-Header "Adding CName Record for App"
$dcInfo = Get-ADDomainController
Do
{
Write-Host "Waiting for Web App Service, hold tight..."
$appIpaddress= kubectl -n $appNamespace get svc "web-app-service" -o jsonpath="{.status.loadBalancer.ingress[0].ip}"
Start-Sleep -Seconds 5
} while ($null -eq $appIpaddress)
Add-DnsServerResourceRecord -ComputerName $dcInfo.HostName -ZoneName $dcInfo.Domain -A -Name "$CName-$sqlInstance" -AllowUpdateAny -IPv4Address $appIpaddress -TimeToLive 01:00:00 -AgeRecord
Add-DnsServerResourceRecordCName -Name $CName -ComputerName $dcInfo.HostName -HostNameAlias "$CName-$sqlInstance.jumpstart.local" -ZoneName jumpstart.local -TimeToLive 00:05:00

Do {
Write-Host "Waiting for Web App pod, hold tight..."
Expand All @@ -139,7 +111,7 @@ Do {
$shortcutLocation = "$Env:Public\Desktop\Bookstore.lnk"
$wScriptShell = New-Object -ComObject WScript.Shell
$shortcut = $wScriptShell.CreateShortcut($shortcutLocation)
$shortcut.TargetPath = "https://$certdns"
$shortcut.TargetPath = "http://$CName.jumpstart.local"
$shortcut.IconLocation="$Env:ArcBoxIconDir\bookstore.ico, 0"
$shortcut.WindowStyle = 3
$shortcut.Save()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ var networkInterfaceName = '${vmName}-NIC'
var osDiskType = 'Premium_LRS'
var k3sControlPlane = 'true' // deploy single-node k3s control plane
var diskSize = (flavor == 'DataOps') ? 512 : 64
var numberOfIPAddresses = (flavor == 'DataOps') ? 7 : 5 // The number of IP addresses to create
var numberOfIPAddresses = (flavor == 'DataOps') ? 8 : 5 // The number of IP addresses to create

// Create multiple public IP addresses if deployBastion is false
resource publicIpAddresses 'Microsoft.Network/publicIpAddresses@2022-01-01' = [for i in range(1, numberOfIPAddresses): {
Expand Down