paginateGetLogEvents should guarantee sane behavior or throw an error explaining how to call it #6631
Open
3 of 4 tasks
Comments
jedwards1211
added
bug
jedwards1211
changed the title
zshzbh
added
p2
|
Hey @jedwards1211 , Thanks for the feedback! Have you tried to limit the page size? How many logs is expected? I don't see this behavior but maybe there's not a lot of the logs in my target log stream. This is the code I have : import { paginateGetLogEvents, CloudWatchLogsClient} from "@aws-sdk/client-cloudwatch-logs";
async function getAllLogEvents(logGroupName, logStreamName, startTime, endTime) {
const client = new CloudWatchLogsClient({ region: "us-west-2" });
const paginator = paginateGetLogEvents(
{
client,
stopOnSameToken: true,
// startFromHead: true, // I got the same result if set startFromHead to false
// pageSize: 1, // adjust as needed
},
{
logGroupName: logGroupName.replace(/:\*$/, ""),
logStreamName,
startTime: startTime.getTime(),
endTime: endTime.getTime(),
}
);
const allEvents = [];
try {
for await (const page of paginator) {
if (page.events) {
//console.log("page-----",page.events)
allEvents.push(...page.events);
}
}
return allEvents;
} catch (error) {
console.error("Error fetching logs:", error);
throw error;
}
}
const startTime = new Date("2024-11-14T00:00:00Z");
const endTime = new Date("2024-11-16T00:00:00Z");
const res = await getAllLogEvents("test","day1",startTime, endTime)
console.log(res)And it returns all logs in the log stream - |
|
I tried with a real log group & log stream. The first couple results I got with startFromHead: true - The logs without startFromHead - The pagination doesn't end unexpectedly. But the sequence of the logs is different. Could you please share more information on :
|
github-actions
bot
removed
the
response-requested
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Checkboxes for prior research
Describe the bug
Right now, we have to manually set
startFromHead: truewhen callingpaginateGetLogEvents, or the pagination ends unexpectedly.#5559 (comment)
This kind of thing is a really irritating waste of time, it should be improved.
paginateGetLogEventsshould set whatever default options necessary to ensure sane behavior or throw an error explaining how to call it in a way that actually worksRegression Issue
SDK version number
@aws-sdk/[email protected]
Which JavaScript Runtime is this issue in?
Node.js
Details of the browser/Node.js/ReactNative version
Node 20.10.0
Reproduction Steps
Observed Behavior
the loop ends after getting one page
Expected Behavior
the loop goes through all pages between
startTimeandendTimePossible Solution
If the underlying service can't be improved then
paginateGetLogEventsshould set a default value ofstartFromHead: truewhen none is provided, so that it can behave in a sane manner. If paginating backward from most to least recent withstartFromHead: false/no value can't work, thenpaginateGetLogEventsshould just throw an error explaining it that option doesn't work.Additional Information/Context
No response
The text was updated successfully, but these errors were encountered: