Skip to content

Commit

Permalink
Update hms.md
Browse files Browse the repository at this point in the history
  • Loading branch information
alberttwong authored Jun 4, 2024
1 parent b58175b commit 193fe16
Showing 1 changed file with 42 additions and 0 deletions.
42 changes: 42 additions & 0 deletions website/docs/hms.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,16 @@ cd $HUDI_HOME/hudi-sync/hudi-hive-sync
--table <tableName>
```

```shell md title="example for s3 with HMS"
./run_sync_tool.sh \
--metastore-uris 'thrift://hive-metastore:9083' \
--partitioned-by city \
--base-path 's3a://warehouse/people' \
--database hudi_db \
--table people \
--sync-mode hms
```

:::note Note:
Replace `file:///path/to/source/data` to appropriate source data path
if you have your source table in S3/GCS/ADLS i.e.
Expand Down Expand Up @@ -218,6 +228,38 @@ using query engines like `Presto` and/or `Trino`. Check out the guides for query
SELECT * FROM iceberg_db.<table_name>;
```

```sql md title="sql for Iceberg, Hudi and Delta Lake in S3"
select * from hudi.hudi_db.people;
select * from delta.delta_db.people;
select * from iceberg.iceberg_db.people;
```

In the example docker image, you can find the catalog configurations in `/etc/trino/catalog`
```shell md title="Trino Catalog with S3"
trino@trino:/$ cat /etc/trino/catalog/hudi.properties
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
connector.name=hudi
hive.metastore.uri=thrift://hive-metastore:9083
hive.s3.aws-access-key=admin
hive.s3.aws-secret-key=password
hive.s3.endpoint=http://minio:9000
```

</TabItem>
</Tabs>

Expand Down

0 comments on commit 193fe16

Please sign in to comment.