Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rescue extract-xiso from oblivion #80

Open
wants to merge 59 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
a5857a3
Define exiso_warn macro, set 's_warned'
rapperskull Mar 4, 2023
90f0945
Rewrite traverse_xiso() to use recursion
rapperskull Mar 4, 2023
01f8bb1
Fix some warnings
rapperskull Mar 4, 2023
0b1804f
Replace all malloc+sprintf with asprintf
rapperskull Mar 4, 2023
c07d089
Rewrite path separation to use strrchr and avoid empty for loops
rapperskull Mar 5, 2023
b04abd0
Move all newlines to beginning of strings
rapperskull Mar 5, 2023
2e81f67
Fix: don't count empty subdirectories as files
rapperskull Mar 5, 2023
093f36a
Fix memory leak in write_tree
rapperskull Mar 5, 2023
c14ddce
Check before free
rapperskull Mar 6, 2023
4759c86
Add new strategy to list files in directory
rapperskull Mar 6, 2023
b9a36ea
Many fixes for directories with size 0
rapperskull Mar 6, 2023
2fda995
Fix integer overflow in write_tree
rapperskull Mar 6, 2023
54375af
Various changes for legibility
rapperskull Mar 7, 2023
ccc5373
Fixed bug that would not skip files >= 4 GiB
rapperskull Mar 4, 2023
8f4bb08
Fix a bug that would cause unnecessary overlapping writes for xbe fil…
rapperskull Mar 11, 2023
49e0f25
Fix some data types, reduce unnecessarly large buffer
rapperskull Mar 14, 2023
9c636dd
Fix lseek with wrapper that returns error when trying to seek past fi…
rapperskull Mar 14, 2023
5216be4
Use realpath/_fullpath to get absolute path of output dir
rapperskull Mar 15, 2023
9be2806
Fix compilation under FreeBSD, OpenBSD, MinGW and Cygwin/MSYS2, add s…
rapperskull Mar 11, 2023
a029894
Swallow the semicolon in function-like macros
rapperskull Mar 17, 2023
80e7d5d
Simplify free_dir_node_avl
rapperskull Mar 17, 2023
6928e89
Rewrite usage macro to include exit and print to stdout when using -h…
rapperskull Mar 17, 2023
3dd3297
Fix calculation of total size and files number across isos
rapperskull Mar 18, 2023
c9dacc3
Fix typo and spacing, use tabulation to align output
rapperskull Mar 19, 2023
c76c66a
Add const qualifiers where possible
rapperskull Mar 20, 2023
481ab8d
Simplify main a bit
rapperskull Mar 21, 2023
13e88af
Add alternative XGD2 offset
rapperskull Mar 27, 2023
64e7003
Add support for Unicode characters
rapperskull Mar 27, 2023
37fa5e7
Fix rewrite mode skipping some files
rapperskull Mar 28, 2023
ae06b0c
Compare as unsigned in avl_compare_key
rapperskull Mar 28, 2023
53fe37a
Preserve file attributes when rewriting
rapperskull Mar 28, 2023
fe885c6
Use less memory when reading the file names
rapperskull Mar 28, 2023
2debfbb
Fix potential memory leak
rapperskull Mar 28, 2023
fd97a47
Improve speed and memory usage when rewriting or creating by using a …
rapperskull Mar 28, 2023
f03952e
Use stdbool.h, since we're compiling under C99 anyways
rapperskull Mar 29, 2023
7cae2b4
Use Windows-1252 locale if available on the system and avoid UTF-8 en…
rapperskull Mar 29, 2023
c9f5b4b
Replace nil with standard NULL, remove unused defines
rapperskull Mar 29, 2023
a56c7b2
Simplify locale selection code
rapperskull Mar 29, 2023
0e6eeaf
Add const qualifiers
rapperskull Mar 30, 2023
36a9a1f
Rewrite boyer moore implementation
rapperskull Apr 1, 2023
1bc46b4
Fix multiple ISOs extraction (fixes XboxDev/extract-xiso#85)
rapperskull Sep 4, 2023
779b0b6
Remove unnecessary seek
rapperskull Oct 3, 2023
6384074
Fix Release CI
rapperskull Oct 3, 2023
83f5901
Properly convert CP1252 filenames to uppercase
rapperskull Feb 27, 2024
4ce46e3
Fix CI build by updating FreeBSD to 13.2 and OpenBSD to 7.4
rapperskull Feb 27, 2024
bca856c
Update upload-artifact and download-artifact to v4
rapperskull Feb 27, 2024
6113b15
Allow the path to file to include symlinks
rapperskull Feb 27, 2024
2334785
Replace getopt and aspritf implementations
rapperskull Feb 28, 2024
1d92f4f
Fix CI by checking out submodules
rapperskull Feb 28, 2024
59bfde6
Replace dirent implementation with one capable of detecting symlinks
rapperskull Feb 29, 2024
9a0f38c
Include the licenses of our dependencies in the Windows release
rapperskull Feb 29, 2024
21fe68f
Make all functions static
rapperskull Mar 2, 2024
1013c56
Skip $SystemUpdate directory during tree parsing, not writing
rapperskull Mar 3, 2024
eb47daa
Remove unnecessary variable
rapperskull Mar 3, 2024
ae5f197
Update dependecy
rapperskull Mar 3, 2024
1563302
Use unsigned char for buffers
rapperskull Sep 16, 2024
1596f94
Update CI for FreeBSD 13.4, OpenBSD 7.5 and NetBSD 9.4
rapperskull Sep 16, 2024
6e3ee44
Handle XISO timestamp explicitly
rapperskull Sep 17, 2024
05582e1
Remove unnecessary parameter from extract_file
rapperskull Sep 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/data/BSD.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# CMAKE_SYSTEM_NAME, TARGET and SYSROOT_PATH must be set

set(CMAKE_SYSTEM_NAME ${CMAKE_SYSTEM_NAME})

set(CMAKE_SYSTEM_PROCESSOR x86_64)

set(CMAKE_C_COMPILER clang)
set(CMAKE_C_COMPILER_TARGET ${TARGET})
set(CMAKE_AR ar)
set(CMAKE_AR_TARGET ${TARGET})
set(CMAKE_LD lld)
set(CMAKE_LD_TARGET ${TARGET})

set(CMAKE_SYSROOT ${SYSROOT_PATH})

# these variables tell CMake to avoid using any binary it finds in
# the sysroot, while picking headers and libraries exclusively from it
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
228 changes: 140 additions & 88 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
echo "BUILD_TAG=$BUILD_TAG" >> $GITHUB_ENV
echo -n $BUILD_TAG > tag
- name: Upload artifacts
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
with:
name: tag
path: tag
Expand All @@ -43,28 +43,40 @@ jobs:
platform: x64
artifact_os: Win64
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
with:
submodules: 'true'
- name: CMake generate
run: |
mkdir build && cd build
cmake .. -A ${{ matrix.platform }}
- name: Build
working-directory: build
run: cmake --build . --config ${{ matrix.configuration }} -j $env:NUMBER_OF_PROCESSORS
- name: Copy licenses
run: |
mkdir LICENSES
for /D %%x in (deps/*) do (
mkdir LICENSES\%%x
copy deps\%%x\LICENSE* LICENSES\%%x
)
shell: cmd
- name: Prepare artifacts
run: |
mkdir artifacts
mv -vb build\${{ matrix.configuration }}\extract-xiso.exe, LICENSE.TXT artifacts
- uses: actions/upload-artifact@v2
mv -vb build\${{ matrix.configuration }}\extract-xiso.exe, LICENSE.TXT, LICENSES artifacts
- uses: actions/upload-artifact@v4
with:
name: extract-xiso_${{ matrix.artifact_os }}_${{ matrix.configuration }}
name: extract-xiso-${{ matrix.artifact_os }}-${{ matrix.configuration }}
path: artifacts

build-linux:
runs-on: ubuntu-latest
needs: Init
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
with:
submodules: 'true'
- name: CMake generate
run: |
mkdir build && cd build
Expand All @@ -76,16 +88,18 @@ jobs:
run: |
mkdir artifacts
mv -v build/extract-xiso LICENSE.TXT artifacts
- uses: actions/upload-artifact@v2
- uses: actions/upload-artifact@v4
with:
name: extract-xiso_${{ runner.os }}
name: extract-xiso-linux
path: artifacts

build-macos:
runs-on: macos-latest
needs: Init
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
with:
submodules: 'true'
- name: CMake generate
run: |
mkdir build && cd build
Expand All @@ -97,105 +111,143 @@ jobs:
run: |
mkdir artifacts
mv -v build/extract-xiso LICENSE.TXT artifacts
- uses: actions/upload-artifact@v2
- uses: actions/upload-artifact@v4
with:
name: extract-xiso-macos
path: artifacts

build-freebsd:
runs-on: ubuntu-latest
needs: Init
env:
SYSROOT_PATH: /opt/cross-freebsd-13
steps:
- uses: actions/checkout@v4
with:
submodules: 'true'
- name: FreeBSD toolchain setup
run: |
sudo apt-get -qq install clang lld && cd /tmp && \
wget -nv http://ftp.plusline.de/FreeBSD/releases/amd64/13.4-RELEASE/base.txz && \
mkdir -p $SYSROOT_PATH && cd $SYSROOT_PATH && \
tar -xf /tmp/base.txz ./lib/ ./usr/lib/ ./usr/include/ && \
cd $SYSROOT_PATH/usr/lib && \
find . -xtype l | xargs ls -l | grep ' /lib/' | awk -v sysroot="$SYSROOT_PATH" '{print "ln -sf " sysroot $11 " " $9}' | /bin/sh && \
rm -f /tmp/base.txz
- name: CMake generate
run: |
mkdir build && cd build
cmake -DCMAKE_TOOLCHAIN_FILE=../.github/data/BSD.cmake -DCMAKE_SYSTEM_NAME=FreeBSD -DTARGET=x86_64-pc-freebsd -DSYSROOT_PATH=$SYSROOT_PATH ..
- name: Build
working-directory: build
run: cmake --build . -j $(nproc --all)
- name: Prepare artifacts
run: |
mkdir artifacts
mv -v build/extract-xiso LICENSE.TXT artifacts
- uses: actions/upload-artifact@v4
with:
name: extract-xiso-freebsd
path: artifacts

build-openbsd:
runs-on: ubuntu-latest
needs: Init
env:
SYSROOT_PATH: /opt/cross-openbsd-7
steps:
- uses: actions/checkout@v4
with:
submodules: 'true'
- name: OpenBSD toolchain setup
run: |
sudo apt-get -qq install clang lld && cd /tmp && \
wget -nv https://cdn.openbsd.org/pub/OpenBSD/7.5/amd64/base75.tgz \
https://cdn.openbsd.org/pub/OpenBSD/7.5/amd64/comp75.tgz && \
mkdir -p $SYSROOT_PATH && cd $SYSROOT_PATH && \
tar -xf /tmp/base75.tgz ./usr/lib/ ./usr/include/ && \
tar -xf /tmp/comp75.tgz ./usr/lib/ ./usr/include/ && \
rm -f /tmp/base75.tgz /tmp/comp75.tgz
- name: CMake generate
run: |
mkdir build && cd build
cmake -DCMAKE_TOOLCHAIN_FILE=../.github/data/BSD.cmake -DCMAKE_SYSTEM_NAME=OpenBSD -DTARGET=x86_64-pc-openbsd -DSYSROOT_PATH=$SYSROOT_PATH ..
- name: Build
working-directory: build
run: cmake --build . -j $(nproc --all)
- name: Prepare artifacts
run: |
mkdir artifacts
mv -v build/extract-xiso LICENSE.TXT artifacts
- uses: actions/upload-artifact@v4
with:
name: extract-xiso-openbsd
path: artifacts

build-netbsd:
runs-on: ubuntu-latest
needs: Init
env:
SYSROOT_PATH: /opt/cross-netbsd-9
steps:
- uses: actions/checkout@v4
with:
submodules: 'true'
- name: NetBSD toolchain setup
run: |
sudo apt-get -qq install clang lld && cd /tmp && \
wget -nv https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.4/amd64/binary/sets/base.tar.xz \
https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.4/amd64/binary/sets/comp.tar.xz && \
mkdir -p $SYSROOT_PATH && cd $SYSROOT_PATH && \
tar -xf /tmp/base.tar.xz ./usr/lib/ ./usr/include/ && \
tar -xf /tmp/comp.tar.xz ./usr/lib/ ./usr/include/ && \
rm -f /tmp/base.tar.xz /tmp/comp.tar.xz
- name: CMake generate
run: |
mkdir build && cd build
cmake -DCMAKE_TOOLCHAIN_FILE=../.github/data/BSD.cmake -DCMAKE_SYSTEM_NAME=NetBSD -DTARGET=x86_64-pc-netbsd -DSYSROOT_PATH=$SYSROOT_PATH ..
- name: Build
working-directory: build
run: cmake --build . -j $(nproc --all)
- name: Prepare artifacts
run: |
mkdir artifacts
mv -v build/extract-xiso LICENSE.TXT artifacts
- uses: actions/upload-artifact@v4
with:
name: extract-xiso_${{ runner.os }}
name: extract-xiso-netbsd
path: artifacts

Release:
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
runs-on: ubuntu-latest
needs: [build-windows, build-linux, build-macos]
env:
BUILD_TAG:
needs: [build-windows, build-linux, build-macos, build-freebsd, build-openbsd, build-netbsd]
steps:
- name: Download artifacts
uses: actions/download-artifact@v2
uses: actions/download-artifact@v4
with:
path: dist
- name: Create archives
run: |
pushd dist/extract-xiso_Win32_Release
zip -r ../extract-xiso-win32-release.zip *
popd
pushd dist/extract-xiso_Win32_Debug
zip -r ../extract-xiso-win32-debug.zip *
popd
###
# Comment out 64 bit builds as they are currently broken
###
# pushd dist/extract-xiso_Win64_Release
# zip -r ../extract-xiso-win64-release.zip *
# popd
# pushd dist/extract-xiso_Win64_Debug
# zip -r ../extract-xiso-win64-debug.zip *
# popd
# pushd dist/extract-xiso_macOS
# zip -r ../extract-xiso-macos.zip *
# popd
for dir in dist/extract-xiso-*/
do
pushd ${dir}
zip_file="../$(basename ${dir} | tr '[:upper:]' '[:lower:]').zip"
echo "Create ${zip_file}"
zip -r ${zip_file} *
popd
done
- name: Get package info
run: |
echo "BUILD_TAG=$(cat dist/tag/tag)" >> $GITHUB_ENV
- name: Create release
id: create_release
uses: actions/create-release@v1
uses: softprops/action-gh-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ env.BUILD_TAG }}
release_name: ${{ env.BUILD_TAG }}
draft: false
prerelease: false
- name: Upload release assets (Win32 build)
id: upload-release-assets-win32-release
uses: actions/[email protected]
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_name: extract-xiso-win32-release.zip
asset_path: dist/extract-xiso-win32-release.zip
asset_content_type: application/zip
- name: Upload release assets (Win32 debug build)
id: upload-release-assets-win32-debug
uses: actions/[email protected]
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_name: extract-xiso-win32-debug.zip
asset_path: dist/extract-xiso-win32-debug.zip
asset_content_type: application/zip
###
# Comment out 64 bit builds as they are currently broken
###
# - name: Upload release assets (Win64 build)
# id: upload-release-assets-win64-release
# uses: actions/[email protected]
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# with:
# upload_url: ${{ steps.create_release.outputs.upload_url }}
# asset_name: extract-xiso-win64-release.zip
# asset_path: dist/extract-xiso-win64-release.zip
# asset_content_type: application/zip
# - name: Upload release assets (Win64 debug build)
# id: upload-release-assets-win64-debug
# uses: actions/[email protected]
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# with:
# upload_url: ${{ steps.create_release.outputs.upload_url }}
# asset_name: extract-xiso-win64-debug.zip
# asset_path: dist/extract-xiso-win64-debug.zip
# asset_content_type: application/zip
# - name: Upload release assets (macOS build)
# id: upload-release-assets-macos-release
# uses: actions/[email protected]
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# with:
# upload_url: ${{ steps.create_release.outputs.upload_url }}
# asset_name: extract-xiso-macos.zip
# asset_path: dist/extract-xiso-macos.zip
# asset_content_type: application/zip
fail_on_unmatched_files: true
files: dist/*.zip
9 changes: 9 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[submodule "deps/getopt_port"]
path = deps/getopt_port
url = https://github.com/kimgr/getopt_port.git
[submodule "deps/asprintf"]
path = deps/asprintf
url = https://github.com/eiszapfen2000/asprintf.git
[submodule "deps/dirent"]
path = deps/dirent
url = https://github.com/tronkko/dirent.git
54 changes: 53 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 3.0)
cmake_minimum_required(VERSION 3.20)
project(extract-xiso)

string(TOUPPER __${CMAKE_SYSTEM_NAME}__ TARGET_OS)
Expand All @@ -7,6 +7,58 @@ set(SOURCE_FILES
extract-xiso.c
)

set(DEPS_DIR ${CMAKE_CURRENT_SOURCE_DIR}/deps)
set(DIRENT_DIR ${DEPS_DIR}/dirent)
set(GETOPT_DIR ${DEPS_DIR}/getopt_port)
set(ASPRINTF_DIR ${DEPS_DIR}/asprintf)

if(MSVC)
set(CMAKE_C_FLAGS_RELEASE_INIT "/O2")
add_compile_options(/W4 /utf-8)
add_compile_options("$<$<CONFIG:DEBUG>:/DDEBUG=1>")
add_definitions(-D_CRT_NONSTDC_NO_WARNINGS) # POSIX function names
add_definitions(-D_CRT_SECURE_NO_WARNINGS) # Unsafe CRT Library functions
else()
if(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE Release)
endif()
set(CMAKE_C_FLAGS_DEBUG "-g -DDEBUG=1")
set(CMAKE_C_FLAGS_RELEASE "-O2 -DNDEBUG")
add_compile_options(-Wall -Wextra -Wpedantic -Wno-comment)
endif()

add_executable(extract-xiso ${SOURCE_FILES})
target_compile_definitions(extract-xiso PRIVATE ${TARGET_OS})
if(CMAKE_C_BYTE_ORDER MATCHES "^$")
set(CMAKE_C_BYTE_ORDER UNK_ENDIAN)
endif()
target_compile_definitions(extract-xiso PRIVATE CMAKE_ENDIANNESS=${CMAKE_C_BYTE_ORDER})
set_property(TARGET extract-xiso PROPERTY C_STANDARD 99)
set_property(TARGET extract-xiso PROPERTY C_EXTENSIONS OFF)

if(WIN32)
add_library(dirent INTERFACE)
target_include_directories(dirent INTERFACE ${DIRENT_DIR}/include)

target_link_libraries(extract-xiso dirent)

add_library(getopt STATIC ${GETOPT_DIR}/getopt.c)
target_sources(getopt PUBLIC ${GETOPT_DIR}/getopt.h)
target_include_directories(getopt PUBLIC ${GETOPT_DIR})
set_property(TARGET getopt PROPERTY C_STANDARD 99)
set_property(TARGET getopt PROPERTY C_EXTENSIONS OFF)

target_link_libraries(extract-xiso getopt)
endif()

if(MSVC)
add_library(asprintf STATIC ${ASPRINTF_DIR}/asprintf.c)
target_sources(asprintf PUBLIC ${ASPRINTF_DIR}/asprintf.h)
target_include_directories(asprintf PUBLIC ${ASPRINTF_DIR})
set_property(TARGET asprintf PROPERTY C_STANDARD 99)
set_property(TARGET asprintf PROPERTY C_EXTENSIONS OFF)

target_link_libraries(extract-xiso asprintf)
endif()

install(TARGETS extract-xiso RUNTIME DESTINATION "${CMAKE_INSTALL_PREFIX}/bin")
Loading
Loading