Merge remote-tracking branch 'origin/master' into task/modify-lakefs-…

…python-docs-7120

# Conflicts:
#	clients/python-wrapper/setup.py

.github/actions/bootstrap-test-lakefs/action.yaml

@@ -24,7 +24,7 @@ runs:
       shell: bash
       run: tar -xf /tmp/generated.tar.gz
     - name: Configure AWS Credentials
-      uses: aws-actions/configure-aws-credentials@v1-node16
+      uses: aws-actions/configure-aws-credentials@v4
       with:
         aws-access-key-id: ${{ env.AWS_ACCESS_KEY_ID }}
         aws-secret-access-key: ${{ env.AWS_SECRET_ACCESS_KEY }}
@@ -36,8 +36,8 @@ runs:
       env:
         LAKEFS_STATS_ENABLED: "false"
         LAKEFS_GATEWAYS_S3_DOMAIN_NAME: s3.docker.lakefs.io:8000
+      shell: bash
+      working-directory: ${{ inputs.compose-directory }}
       run: |
         [[ -z "${{ inputs.compose-file }}" ]] && flags="" || flags="-f ${{ inputs.compose-file }}"
         docker-compose ${flags} up ${{ inputs.compose-flags }}
-      shell: bash
-      working-directory: ${{ inputs.compose-directory }}

.github/workflows/docker-publish-lakefs-rclone-export.yaml

@@ -32,7 +32,7 @@ jobs:
         id: version
 
       - name: Configure AWS Credentials
-        uses: aws-actions/configure-aws-credentials@v3
+        uses: aws-actions/configure-aws-credentials@v4
         with:
           aws-region: us-east-1
           role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}

.github/workflows/docker-publish.yaml

@@ -56,7 +56,7 @@ jobs:
           GOLANGCI_LINT_FLAGS: --out-format github-actions
 
       - name: Configure AWS Credentials
-        uses: aws-actions/configure-aws-credentials@v3
+        uses: aws-actions/configure-aws-credentials@v4
         with:
           aws-region: us-east-1
           role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}

.github/workflows/esti.yaml

@@ -83,7 +83,7 @@ jobs:
         run: tar -xf /tmp/generated.tar.gz
 
       - name: Configure AWS Credentials
-        uses: aws-actions/configure-aws-credentials@v3
+        uses: aws-actions/configure-aws-credentials@v4
         with:
           aws-region: us-east-1
           role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
@@ -117,7 +117,7 @@ jobs:
     if: needs.check-secrets.outputs.secretsavailable == 'true'
     steps:
       - name: Configure AWS credentials
-        uses: aws-actions/configure-aws-credentials@v3
+        uses: aws-actions/configure-aws-credentials@v4
         with:
           role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
           aws-region: us-east-1
@@ -240,7 +240,7 @@ jobs:
         uses: actions/checkout@v4
 
       - name: Configure AWS Credentials
-        uses: aws-actions/configure-aws-credentials@v3
+        uses: aws-actions/configure-aws-credentials@v4
         with:
           aws-region: us-east-1
           role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}

.github/workflows/goreleaser.yaml

@@ -103,7 +103,7 @@ jobs:
           go-version: "1.21.4"
 
       - name: Configure AWS Credentials
-        uses: aws-actions/configure-aws-credentials@v3
+        uses: aws-actions/configure-aws-credentials@v4
         with:
           aws-region: us-east-1
           role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}

.github/workflows/publish-hadoop-lakefs.yaml

@@ -49,7 +49,7 @@ jobs:
         run: sed -i.bak 's/<version>.*<\/version><!--MARKER.*/<version>'${{ steps.version.outputs.tag }}'<\/version>/' pom.xml
 
       - name: Configure AWS Credentials
-        uses: aws-actions/configure-aws-credentials@v3
+        uses: aws-actions/configure-aws-credentials@v4
         with:
           aws-region: us-east-1
           role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}

.github/workflows/publish-python-wrapper-client.yaml

@@ -10,22 +10,11 @@ jobs:
     - name: Checkout
       uses: actions/checkout@v4
 
-    - name: Python Build and Make Package
-      run: make package-python-wrapper
-
-    - name: Publish Distribution 📦 to PyPI
-      uses: pypa/gh-action-pypi-publish@release/v1
-      with:
-        user: __token__
-        password: ${{ secrets.PYPI_API_TOKEN }}
-        packages-dir: clients/python-wrapper/dist/
-        verbose: true
-
     - name: Extract docs version
       working-directory: ./clients/python-wrapper/
       shell: bash
       run: |
-          echo "tag=$(python setup.py --version)" >> $GITHUB_OUTPUT
+        echo "tag=$(python setup.py --version)" >> $GITHUB_OUTPUT
       id: docver
 
     - name: Set up Python 3.9
@@ -34,7 +23,7 @@ jobs:
         python-version: '3.9'
         cache: 'pip'
         cache-dependency-path: 'clients/python-wrapper/requirements.txt'
-    
+
     - name: Generate and Build Documentation
       run: |
         pip install -r clients/python-wrapper/requirements.txt
@@ -50,3 +39,15 @@ jobs:
         destination_folder: /
         user_email: '[email protected]'
         user_name: 'python-docs-action'
+
+    - name: Python Build and Make Package
+      run: make package-python-wrapper
+
+    - name: Publish Distribution 📦 to PyPI
+      uses: pypa/gh-action-pypi-publish@release/v1
+      with:
+        user: __token__
+        password: ${{ secrets.PYPI_API_TOKEN }}
+        packages-dir: clients/python-wrapper/dist/
+        verbose: true
+

CHANGELOG.md

@@ -1,5 +1,18 @@
 # Changelog
 
+# v1.3.1
+
+:bug: Bugs fixed:
+
+- Fix: User with permission to write actions can impersonate another user when auth token is configured in environment variable
+  ([GHSA-26hr-q2wp-rvc5](https://github.com/treeverse/lakeFS/security/advisories/GHSA-26hr-q2wp-rvc5))
+- Fix: S3 Gateway block unsupported S3 operations (#7028)
+- Fix: Better error handling on hook error (#7081)
+- Fix: Upload object without specify content type (#7130)
+- Fix: UI notebook preview fix colors (#7141)
+- Fix: Match blockstore reader hash function (#7099) (thanks @hunjixin)
+- UI improve load by cache and split of embedded content (#7132,#7135)
+
 # v1.3.0
 
 :new: What's new:

clients/python-wrapper/CHANGELOG.md

@@ -0,0 +1,5 @@
+# Changelog
+
+## v0.1.0
+What's new:
+- First official release!

clients/python-wrapper/README.md

@@ -1,4 +1,4 @@
-# lakeFS High-Level Python SDK (Pre-release Beta)
+# lakeFS High-Level Python SDK
 
 lakeFS High Level SDK for Python, provides developers with the following features:
 1. Simpler programming interface with less configuration

clients/python-wrapper/lakefs/object.py

@@ -146,7 +146,10 @@ def writelines(self, lines: List[AnyStr]) -> None:
         raise io.UnsupportedOperation
 
     def __next__(self) -> AnyStr:
-        return self.readline()
+        line = self.readline()
+        if len(line) == 0:
+            raise StopIteration
+        return line
 
     def __iter__(self) -> Iterator[AnyStr]:
         return self

clients/python-wrapper/setup.py

@@ -1,7 +1,7 @@
 from setuptools import setup, find_packages
 
 NAME = "lakefs"
-VERSION = "0.1.0-beta.3"
+VERSION = "0.1.0"
 # To install the library, run the following
 #
 # python setup.py install

clients/python-wrapper/tests/integration/test_object.py

@@ -306,3 +306,26 @@ def test_readline_partial_line_buffer(setup_repo):
             assert read == expected
 
         assert reader.read() == ""
+
+
+def test_write_read_csv(setup_repo):
+    _, repo = setup_repo
+    columns = ["ID", "Name", "Email"]
+    sample_data = [
+        ['1', "Alice", "[email protected]"],
+        ['2', "Bob", "[email protected]"],
+        ['3', "Carol", "[email protected]"],
+    ]
+    obj = repo.branch("main").object(path="csv/sample_data.csv")
+
+    with obj.writer(mode='w', pre_sign=False, content_type="text/csv") as fd:
+        writer = csv.writer(fd)
+        writer.writerow(columns)
+        for row in sample_data:
+            writer.writerow(row)
+
+    for i, row in enumerate(csv.reader(obj.reader(mode='r'))):
+        if i == 0:
+            assert row == columns
+        else:
+            assert row == sample_data[i - 1]

docs/reference/security/sso.md

@@ -103,8 +103,6 @@ Important: new app registrations are hidden to users by default. When you are re
 ### Add a secret
 Sometimes called an application password, a client secret is a string value your app can use in place of a certificate to identity itself.
 
-Client secrets are considered less secure than certificate credentials. Application developers sometimes use client secrets during local app development because of their ease of use. However, you should use certificate credentials for any of your applications that are running in production.
-
 Steps:
 1. In the Azure portal, in App registrations, select your application.
 2. Select Certificates & secrets > Client secrets > New client secret.
@@ -120,7 +118,7 @@ A redirect URI is the location where the Microsoft identity platform redirects a
 
 You add and modify redirect URIs for your registered applications by configuring their platform settings.
 
-Enter https://lakefs-cloud.us.auth0.com/login/callback or https://lakefs-cloud.eu.auth0.com/login/callback (depends on your organization data location) as your redirect URI.
+Enter https://lakefs-cloud.us.auth0.com/login/callback as your redirect URI.
 
 Settings for each application type, including redirect URIs, are configured in Platform configurations in the Azure portal. Some platforms, like Web and Single-page applications, require you to manually specify a redirect URI. For other platforms, like mobile and desktop, you can select from redirect URIs generated for you when you configure their other settings.
 
@@ -131,7 +129,12 @@ Steps:
 4. Under Configure platforms, select the web option.
 5. Select Configure to complete the platform configuration.
 
-Once you finish registering lakeFS Cloud with Azure AD, save the **Application (Client) ID**, **Application Secret Value** and send this to Treeverse's team to finish the integration.
+Once you finish registering lakeFS Cloud with Azure AD send the following items to the Treeverse's team:
+1. **Client ID**
+2. **Client Secret**
+3. **Azure AD Domain**
+4. **Identity API Version** (v1 for Azure AD or v2 for Microsoft Identity Platform/Entra) 
+
   </div>
 </div>