Skip to content

Commit

Permalink
[hive] Add license for TezUtil
Browse files Browse the repository at this point in the history
  • Loading branch information
JingsongLi committed Mar 18, 2024
1 parent 48c239d commit f33f932
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 5 deletions.
1 change: 1 addition & 0 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,7 @@ paimon-core/src/main/java/org/apache/paimon/jdbc/JdbcCatalog.java
paimon-core/src/main/java/org/apache/paimon/utils/ZOrderByteUtils.java
paimon-core/src/test/java/org/apache/paimon/utils/TestZOrderByteUtil.java
paimon-hive/paimon-hive-common/src/test/java/org/apache/paimon/hive/TestHiveMetastore.java
paimon-hive/paimon-hive-connector-common/src/main/java/org/apache/paimon/hive/mapred/TezUtil.java
paimon-spark/paimon-spark-common/src/main/antlr4/org.apache.spark.sql.catalyst.parser.extensions/PaimonSqlExtensions.g4
paimon-spark/paimon-spark-common/src/main/java/org/apache/paimon/spark/SparkGenericCatalog.java
paimon-spark/paimon-spark-common/src/main/scala/org/apache/spark/sql/catalyst/analysis/CoerceArguments.scala
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,11 @@

import java.util.Objects;

/**
* Utility class to enrich the JobContext and TaskAttemptContext with the vertex id. copied form
* iceberg.
*/
/* This file is based on source code from the Iceberg Project (http://iceberg.apache.org/), licensed by the Apache
* Software Foundation (ASF) under the Apache License, Version 2.0. See the NOTICE file distributed with this work for
* additional information regarding copyright ownership. */

/** Utility class to enrich the JobContext and TaskAttemptContext with the vertex id. */
public class TezUtil {

private static final String TASK_ATTEMPT_ID_KEY = "mapred.task.id";
Expand Down Expand Up @@ -96,7 +97,7 @@ private TezUtil() {}
/**
* Subclasses {@link TaskAttemptID}. It has two main purposes: 1. Provide a way to append an
* optional vertex id to the Job ID. This is needed because there is a discrepancy between how
* the attempt ID is constructed in the {@link org.apache.tez.mapreduce.output.MROutput} (with
* the attempt ID is constructed in the {@code org.apache.tez.mapreduce.output.MROutput} (with
* vertex ID appended to the end of the Job ID) and how it's available in the mapper (without
* vertex ID) which creates and caches the HiveIcebergRecordWriter object. 2. Redefine the
* equals/hashcode provided by TaskAttemptID so that task type (map or reduce) does not count,
Expand Down

0 comments on commit f33f932

Please sign in to comment.