Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 56: add noop mode #57

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,13 @@ Cron job for deleting old, unused versions of your Function.

This [post](https://lumigo.io/blog/a-serverless-application-to-clean-up-old-deployment-packages/) explains the problem and why we created this app.

## No-Op (no operation, or, dry run) mode

To do dry runs of this app, the environment variable NOOP can be set using the
`NoOp` parameter. NoOp will be considered `true` (no active run) if `NoOp` is:
* a string that is in the array `["true", "t", "yes", "y"]` when lower-cased
* a string that casts to an int >= 1

## Safeguards

To guard against deleting live versions, some safeguards are in place:
Expand Down Expand Up @@ -39,6 +46,7 @@ AutoDeployMyAwesomeLambdaLayer:
SemanticVersion: <enter latest version>
Parameters:
VersionsToKeep: <defaults to 3>
NoOp: <optional, defaults to "", i.e. False (do active run)>
```

To do the same via CloudFormation or the Serverless framework, you need to first add the following `Transform`:
Expand Down
17 changes: 16 additions & 1 deletion functions/clean.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,16 @@ module.exports.handler = async () => {
log.debug("all done");
};

const isEnvTrue = (envVal) => {
if (!envVal){return false;}
const parsedI = parseInt(envVal, 10);
if (! isNaN(parsedI)) {return parsedI >= 1;}
if (["t", "true", "y", "yes"].includes(envVal.toLowerCase())){
return true;
}
return false;
};

const clean = async () => {
if (functions.length === 0) {
functions = await Lambda.listFunctions();
Expand Down Expand Up @@ -38,14 +48,19 @@ const cleanFunc = async (funcArn) => {
versions = _.orderBy(versions, v => parseInt(v), "desc");

const versionsToKeep = parseInt(process.env.VERSIONS_TO_KEEP || "3");
const noop = isEnvTrue(process.env.NOOP);

// drop the most recent N versions
log.debug(`keeping the most recent ${versionsToKeep} versions`);
versions = _.drop(versions, versionsToKeep);

for (const version of versions) {
if (!aliasedVersions.includes(version)) {
await Lambda.deleteVersion(funcArn, version);
if (noop) {
console.log(`NOOP: would have attempted to delete function ${funcArn} version ${version}`);
} else {
await Lambda.deleteVersion(funcArn, version);
}
}
}
};
47 changes: 46 additions & 1 deletion functions/clean.test.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
const { isNull } = require("lodash");
const Lambda = require("./lib/lambda");

console.log = jest.fn();
Expand All @@ -21,8 +22,13 @@ afterEach(() => {
mockDeleteVersion.mockClear();
});

const requireHandler = (versionsToKeep) => {
const requireHandler = (versionsToKeep, noop=null) => {
process.env.VERSIONS_TO_KEEP = versionsToKeep.toString();
if (! isNull(noop)) {
process.env.NOOP = noop.toString();
} else {
delete process.env.NOOP;
}
return require("./clean").handler;
};

Expand All @@ -37,6 +43,45 @@ test("when there are no functions, it does nothing", async () => {
expect(mockDeleteVersion).not.toBeCalled();
});

const noopCases = [
{env: "True", run_expected: false},
{env: "true", run_expected: false},
{env: "yes", run_expected: false},
{env: "T", run_expected: false},
{env: "t", run_expected: false},
{env: "Y", run_expected: false},
{env: "y", run_expected: false},
{env: "1", run_expected: false},
{env: "13", run_expected: false},
{env: "False", run_expected: true},
{env: "Ture", run_expected: true},
{env: "", run_expected: true},
{env: "0", run_expected: true},
{env: "zzzzzz09ijasdflk23l", run_expected: true}
];

test.each(noopCases)(
"Noop prevents deletion %s",
async (test_case) => {

mockListFunctions.mockResolvedValueOnce(["a"]);
mockListVersions.mockResolvedValueOnce(["1", "2", "3"]);
mockListAliasedVersions.mockResolvedValueOnce([]);

const handler = requireHandler(1, test_case.env);
await handler();

if (test_case.run_expected) {
expect(mockDeleteVersion).toHaveBeenCalledTimes(2);
expect(mockDeleteVersion).toBeCalledWith("a", "1");
expect(mockDeleteVersion).toBeCalledWith("a", "2");
} else {
expect(mockDeleteVersion).not.toBeCalled();
}
}

);

test("all unaliased versions of a function is deleted", async () => {
mockListFunctions.mockResolvedValueOnce(["a"]);
mockListVersions.mockResolvedValueOnce(["1", "2", "3"]);
Expand Down
12 changes: 8 additions & 4 deletions functions/lib/lambda.js
Original file line number Diff line number Diff line change
Expand Up @@ -139,10 +139,14 @@ const deleteVersion = async (funcArn, version) => {
};

await retry(
(bail) => lambda
.deleteFunction(params)
.promise()
.catch(bailIfErrorNotRetryable(bail)),
(bail) => {
const res = lambda
.deleteFunction(params)
.promise()
.catch(bailIfErrorNotRetryable(bail));
log.info("deleted Lambda function version", {function: funcArn,});
return res;
},
getRetryConfig((err) => {
log.warn("retrying deleteFunction after error...", { function: funcArn, version }, err);
}));
Expand Down
6 changes: 6 additions & 0 deletions template.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ Resources:
LOG_LEVEL: INFO
VERSIONS_TO_KEEP:
Ref: VersionsToKeep
NOOP:
Ref: NoOp
AWS_NODEJS_CONNECTION_REUSE_ENABLED: 1
Policies:
- Statement:
Expand All @@ -52,3 +54,7 @@ Parameters:
How many versions to keep, even if they are not aliased.
Default: 3
MinValue: 0 # don't keep anything except $Latest
NoOp:
Type: String
Description: Set to `TRUE`, `YES`, or `1` to perform no-operation runs
Default: ""