Potential reownership?

[trulysinclair]

I'm aware you're no longer interested in maintaining this project, however I'd like to peak your interest in adopting this package and working on keeping it stable. I do have personal uses for this, but rather build one from scratch, I figured I'd try to get your permsission to hand off the project to my R&D organization.

I greatly respect the progress you've made, please consider my request.

[1egoman]

Hi! Yes, I'm not - funny you should ask this, but I actually recently built a better version of this same tool (a much more reliable, version that requires much less configuration) that hasn't made its way onto github yet.

Let me do this - I'll push up this new version to a separate, orphan branch on this repository. Take a look at it, and let me know your thoughts and improvements. I'd say for now I'd prefer to keep this repository here, but if you make a few pull requests and things go well I'd consider possibly moving it to a neutral organization we both have access to. Though I don't have an interest in maintaining it, this is a tool I do use myself quite often and I'd like to still be able to have some sort of access to its code so I can fix bugs that effect me.

@TheGrimSilence How does that sound?

[1egoman]

@TheGrimSilence Here's that v2 branch: https://github.com/1egoman/debundle/tree/v2. Take a look around and let me know your thoughts.

[trulysinclair]

Sounds good! I'll look over it tonight when I get home! Thanks for responding 😅

[trulysinclair]

Sorry for the late reply, COVID-19 has been troublesome. But everything looks good! I'll go over it again tonight and take a deeper look.

[1egoman]

No big deal! Take as much time as you need.

[trulysinclair]

Have you considered transitioning to TypeScript? Taking advantage of new standards and abilities without needing Babel?

[1egoman]

I would consider it. I've used typescript a decent amount at work and have had mixed reactions to it - when used lightly, I've found a few benefits, but when (in my opinion, at least) taken to the extreme it results in extremely hard to understand error messages that frustrate me (especially when multiple generics in one function definition are involved, or type-math is overused to try to make an interface as semantically correct as possible). It could just be that I'm not as familiar with the ecosystem, or that I tend to vim and all the language server plugins for vim are bad.

It sounds like though that you have some experience with typescript. If you'd like to try to convert part of the code to start with, that might be an interesting experiment to see how that would work. I would say though that is that's something you want to try, keeping the types as simple as possible is important to me.

[1egoman]

Also, something else that has come to mind recently about this code: right now, it has no tests. I'm not sure exactly what the best way to test it would be - write unit tests around specific sets of circumstances? Try to do more integration-tests where I debundle a bunch of bundles and assert that the output is correct? Maybe a combination of both or some other option I haven't considered? On first thought, I tend to like the integration test idea, since historically, when I had to add new features or ran into bugs that I had to fix, I had a bundle I was working off of that I could just add to a test suite really easily.

That may be another interesting problem to look into if you are interested.

A few other potential ideas worth exploring, if any of them seem like tasks you'd like to try out:

• A problem I had with the previous version of debundle was that people who didn't have a lot of familiarity with the internals of webpack bundles ran into complex scenarios in real-world bundles, and the error messages that were logged were inadequate to diagnose what was going on for those that didn't have a super deep understanding of javascript bundles. I had started to write this ExtendedError abstraction (link: https://github.com/1egoman/debundle/blob/v2/src/utils.js#L7-L22) so that I could somehow document specific error cases with more detail, and this special type of error would handle logging out an error, its description, some context, and a link to a website with even more details (this would probably tie in with that documentation item below). Then, when a user went to the website, it would explain the error in more detail, and possibly provide more context as to what went wrong.

• Currently, the v2 branch only works for webpack bundles. It would be cool if browserify / other bundler's bundles could be unpacked too. My thought for how to do this was to make the WebpackBootstrap class sort of a bundler specific interface, and then there could be versions for other bundlers. Given that webpack's sort of the industry leader, I think that just having webpack for now is fine though. NOTE: Some bundles just can't really be unpacked (or at least, I don't think they can be) because the code they output just has all the modules concatenated together (eg, rollup) and it would be ambiguous what lines of code belong in which module.

• Ideally, I wanted to write up some documentation on how javascript bundling works, and walk through a bundle and all the parts of it - probably a webpack only example is fine for now. Something like "here's the webpackBootstrap section and it orchestrates loading modules", and then "here's where the require function is defined in webpackBootstrap, here's where the entrypoint module is located, etc". Then, I wanted to get a bit into (at least in webpack bundles) the module array and talking about what the module, exports, and require parameters to the each function are used for. And finally, I wanted to touch on how bundle splitting works. I'm not sure how familiar you are with the internals of bundlers, but if you're a decent writer and want an excuse to get more familiar, this might be a worthwhile task.

[trulysinclair]

I spend most of my time with TypeScript so my familiarity is adequate and intermediate. I won't call myself an expert, none of us are 😅. I'll explore various implementations, keeping the typings as simple as possible. The most extreme usage of TypeScript I've seen to date, is Visual Studio Code. The source code is mostly TypeScript, and is the best example of how to implement heavy projects without sacrificing performance.

As far as testing goes, I'll see what I can come up with, I think starting off, the most straightforward way to test it would be integration testing, but I like the dual edged approach, perhaps start with unit tests to make sure the individual components work, and then begin the Integration tests. That way we don't waste valuable test time. Why attempt to test a bundle when we can catch an internal error ahead of time.

I was definitely thinking of exploring various bundle types, I also am working with Webpack bundles. Mine is a headache, as I'm working with real world bundles with no source maps, Webflow. But tonight I'll sit down and get to work on a quick task list, and I'll make a pull request when a single task is done. Progressively tackling this I think is best.

[trulysinclair]

I was also thinking of a far-fetched idea way down the road, but what if we had a way to "recognize" certain packages? Say, we kept track of popular packages like React and based on core methods detected, we are able to effectively name the modules pulled from the bundles.

[1egoman]

I was also thinking of a far-fetched idea way down the road, but what if we had a way to "recognize" certain packages? Say, we kept track of popular packages like React and based on core methods detected, we are able to effectively name the modules pulled from the bundles.

I had thought about that a little too! In the way I had envisioned it, I was thinking that it would be interesting to make some sort of "syntax similarity" method that could pull down the top n most popular packages from npm. Then, the logic could somehow do a brute force check (or maybe something more efficient? This is a just an idea) and once it found a package, all the files could be renamed to match it and it could somehow be tagged with that version.

Let me write up a bit more documentation tonight too, just so you have a bit more to go off of than the quick readme I threw together. In particular, I want to lay out what at least currently the workflow looks like when you want to make tweaks to the bundle's generated output.

[1egoman]

Ok, more documentation has been written. I added a bit more to the "Getting Started" header in the readme: https://github.com/1egoman/debundle/tree/v2#getting-started

[1egoman]

I'm working with real world bundles with no source maps, Webflow.

Hah, yea. It often can be really hard to sift through that sort of code. As I keep reverse engineering different services, I have gotten better at it. It takes practice.

One thing that is totally doable, maybe another idea: I used a package called escope when building this second iteration that is able to parse the AST and find all variable references within a function. It would be cool if there was a setting a user could set on a module that was like "rename all variables to be random words". That would be at least easier to read through than a function that redefines a single letter variable like 5 times throughout its body 😲.

[trulysinclair]

Roughly what I was thinking as well, and the documentation will definitely help! It'll take some trial and error to figure out what best for the recognition. Online fetches would be a little rough, that would make the module dependent on an internet connection. When I did a little work on exploring VS Code, I noticed their editor has an internal directory full of json files, full of each language they support which holds ways to operate with the language. I'm thinking we could hold the same thing for the modules we support and can name.

[trulysinclair]

I agree, I think there's a module for that I've seen. I know Webflow when naming CSS blocks, splotches? Need more coffee. Anyway it names them randomly, I've done wayyyy too much manual reverse engineering sifting through hundreds of thousands of lines recognizing modules like React and React DnD because I'm familiar with their method names and structure. That's how I stumbled on this module 😅.

[1egoman]

@TheGrimSilence Hi, just wanted to check in and see if you've gotten a cnance to take a look around. No big deal if not!

I will say that I've been putting together a small docs site for debundle. I'm finding I have too much to really say for one README.md file!

• I tidied up what I wrote in the readme of that new branch into a bit more of an easier to understand package (http://localhost:8000/docs/reverse-engineering-a-bundle)
• I wrote an iterative sort of tutorial that walks through the process of building a bundler, explaining why different bundler features exist along the way. This is a document I've been wanting to write up fora while now anyway, and this was a good excuse to do it. (http://debundle-docs.surge.sh/docs/building-a-bundler)

Check both out here: http://debundle-docs.surge.sh/docs

Code can be found here: https://github.com/1egoman/debundle/tree/v2-docs-site

[trulysinclair]

Yeah sorry internet is overwhelmed in my area due to lots of layoffs so it's in and out. I made a fork and have been working on the TypeScript conversion but haven't pushed many changes yet. I think my repository is sort of up to date with changes if you want to look at it here

[trulysinclair]

I also added some of my own documentation under a doc folder but I need to push it

[1egoman]

Ah, sorry to hear that. I'll take a look at what you have more in depth when I have some more bandwidth.

[0xdevalias]

I know this thread has been dead for ~3 years.. but I was wondering if anything more ever came of this?

[trulysinclair]

Unfortunately, no. Not on my end at least. After Covid, things kinda fell off in many places for me. Honestly, I have to apologize to @1egoman for forgetting about this project. I'm not sure where things have gone or will go now.

[0xdevalias]

@trulysinclair That's totally understandable.

---

For anyone wanting a modern/maintained tool like this, I was looking through all the tools I could find in this space recently (Ref), and the most promising couple that I came across seemed to be these:

• https://github.com/pionxzh/wakaru

  • Javascript decompiler, unpacker and unminify toolkit

  • Wakaru is the Javascript decompiler for modern frontend. It brings back the original code from a bundled and transpiled source.

  • https://wakaru.vercel.app/
• https://github.com/j4k0xb/webcrack

  • Deobfuscate obfuscator.io, unminify and unpack bundled javascript

  • https://webcrack.netlify.app/ (Playground)
• https://github.com/jehna/humanify

  • Un-minify Javascript code using ChatGPT

  • https://thejunkland.com/blog/using-llms-to-reverse-javascript-minification.html

    • Using LLMs to reverse JavaScript variable name minification