What does msck stands for in Msck repair command
Similar to how fsckstands for filesystem consistency check, msck is Hive’s metastore consistency check.
Similar to how fsckstands for filesystem consistency check, msck is Hive’s metastore consistency check.
Since 0.14, you can use following statement to move table from one database to another in the same metastore: use old_database; alter table table_a rename to new_database.table_a The above statements will also move the table data on hdfs if table_a is a managed table.
Yeah we can do that. You just need to follow below three commands in sequence. Lets say you have a external table test_1 in hive. And you want to rename it test_2 which should point test_2 location not test_1. Then you need to convert this table into Managed table using below command. test_1 -> pointing … Read more
Spark >= 2.4 You can skip zip udf and use arrays_zip function: df.withColumn(“vars”, explode(arrays_zip($”varA”, $”varB”))).select( $”userId”, $”someString”, $”vars.varA”, $”vars.varB”).show Spark < 2.4 What you want is not possible without a custom UDF. In Scala you could do something like this: val data = sc.parallelize(Seq( “””{“userId”: 1, “someString”: “example1”, “varA”: [0, 2, 5], “varB”: [1, 2, … Read more
You cannot add a column with a default value in Hive. You have the right syntax for adding the column ALTER TABLE test1 ADD COLUMNS (access_count1 int);, you just need to get rid of default sum(max_count). No changes to that files backing your table will happen as a result of adding the column. Hive handles … Read more
Yes, here you can use LIMIT. You can try it by the below query: SELECT * FROM employee_list SORT BY salary DESC LIMIT 2
You can use the percentile function to compute the median. Try this: select percentile(cast(age as BIGINT), 0.5) from table_name
datediff(to_date(String timestamp), to_date(String timestamp)) For example: SELECT datediff(to_date(‘2019-08-03’), to_date(‘2019-08-01’)) <= 2;
To get this you can use hive’s property which is TBLPROPERTIES (“skip.header.line.count”=”1”) you can also refer example – CREATE TABLE temp ( name STRING, id INT ) row format delimited fields terminated BY ‘\t’ lines terminated BY ‘\n’ tblproperties(“skip.header.line.count”=”1”);
Looks like problem with your metastore. If you are using the default hive metastore embedded derby. Lock file would be there in case of abnormal exit. if you remove that lock file this issue would be solved rm metastore_db/*.lck