入门客AI创业平台(我带你入门，你带我飞行)
博文笔记

Ambari日常运维操作命令收集

创建时间：2018-03-28 投稿人： 浏览次数：214

Ambari REST API查询相关

  • 获取集群配置信息：http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1?fields=Clusters/desired_configs

  • 特定配置信息(cluster-env)：http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1/configurations?type=cluster-env&tag=version1522220015974073

  • 添加一个新的配置(yarn-env)：使用场景也可能是丢失了yarn-env配置

curl -u admin:admin -H "X-Requested-By:admin" -X PUT "http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1" -d "{
  "Clusters": {
    "desired_config": {
      "type": "yarn-env",
      "tag": "version2",
      "properties": {
        "yarn_heapsize": "1024", 
            "yarn_pid_dir_prefix": "/var/run/hadoop-yarn", 
            "yarn_user_nproc_limit": "65536", 
            "yarn_log_dir_prefix": "/var/log/hadoop-yarn", 
            "yarn_user_nofile_limit": "32768", 
            "service_archive_rela_path": "{"DEFAULT":"hadoop/hadoop-2.7.3-1.2.0.tar.gz", "METRICS_HADOOP_SINK":"ambari-metrics/ambari-metrics-hadoop-sink-2.5.1.0.0.tar.gz"}", 
            "service_version": "2.7.3", 
            "is_supported_yarn_ranger": "false", 
            "nodemanager_heapsize": "1024", 
            "server_role_directory_map": "{"YARN_CLIENT":"yarn_client","RESOURCEMANAGER":"yarn_resourcemanager","NODEMANAGER":"yarn_nodemanager","APP_TIMELINE_SERVER":"yarn_timelineserver", "METRICS_HADOOP_SINK":"ambari_metrics_hadoop_sink"}", 
            "content": "
export HADOOP_YARN_HOME={{component_home}}
export YARN_LOG_DIR={{yarn_log_dir}}
export YARN_PID_DIR={{yarn_pid_dir}}
export HADOOP_LIBEXEC_DIR={{component_libexec_dir}}
export JAVA_HOME={{java_home}}
export JAVA_LIBRARY_PATH="${JAVA_LIBRARY_PATH}:{{hadoop_java_io_tmpdir}}"

# We need to add the EWMA appender for the yarn daemons only;
# however, YARN_ROOT_LOGGER is shared by the yarn client and the
# daemons. This is restrict the EWMA appender to daemons only.
INVOKER="${0##*/}"
if [ "$INVOKER" == "yarn-daemon.sh" ]; then
  export YARN_ROOT_LOGGER=${YARN_ROOT_LOGGER:-INFO,EWMA,RFA}
fi

# User for YARN daemons
export HADOOP_YARN_USER=${HADOOP_YARN_USER:-yarn}

# resolve links - $0 may be a softlink
export YARN_CONF_DIR="${YARN_CONF_DIR:-$HADOOP_YARN_HOME/conf}"

# some Java parameters
export JAVA_HOME={{java_home}}
if [ "$JAVA_HOME" != "" ]; then
  #echo "run java in $JAVA_HOME"
  JAVA_HOME=$JAVA_HOME
fi

if [ "$JAVA_HOME" = "" ]; then
  echo "Error: JAVA_HOME is not set."
  exit 1
fi

JAVA=$JAVA_HOME/bin/java
JAVA_HEAP_MAX=-Xmx1000m

# For setting YARN specific HEAP sizes please use this
# Parameter and set appropriately
YARN_HEAPSIZE={{yarn_heapsize}}

# check envvars which might override default args
if [ "$YARN_HEAPSIZE" != "" ]; then
  JAVA_HEAP_MAX="-Xmx""$YARN_HEAPSIZE""m"
fi

# Resource Manager specific parameters

# Specify the max Heapsize for the ResourceManager using a numerical value
# in the scale of MB. For example, to specify an jvm option of -Xmx1000m, set
# the value to 1000.
# This value will be overridden by an Xmx setting specified in either YARN_OPTS
# and/or YARN_RESOURCEMANAGER_OPTS.
# If not specified, the default value will be picked from either YARN_HEAPMAX
# or JAVA_HEAP_MAX with YARN_HEAPMAX as the preferred option of the two.
export YARN_RESOURCEMANAGER_HEAPSIZE={{resourcemanager_heapsize}}

# Specify the JVM options to be used when starting the ResourceManager.
# These options will be appended to the options specified as YARN_OPTS
# and therefore may override any similar flags set in YARN_OPTS
#export YARN_RESOURCEMANAGER_OPTS=

# Node Manager specific parameters

# Specify the max Heapsize for the NodeManager using a numerical value
# in the scale of MB. For example, to specify an jvm option of -Xmx1000m, set
# the value to 1000.
# This value will be overridden by an Xmx setting specified in either YARN_OPTS
# and/or YARN_NODEMANAGER_OPTS.
# If not specified, the default value will be picked from either YARN_HEAPMAX
# or JAVA_HEAP_MAX with YARN_HEAPMAX as the preferred option of the two.
export YARN_NODEMANAGER_HEAPSIZE={{nodemanager_heapsize}}

# Specify the max Heapsize for the timeline server using a numerical value
# in the scale of MB. For example, to specify an jvm option of -Xmx1000m, set
# the value to 1024.
# This value will be overridden by an Xmx setting specified in either YARN_OPTS
# and/or YARN_TIMELINESERVER_OPTS.
# If not specified, the default value will be picked from either YARN_HEAPMAX
# or JAVA_HEAP_MAX with YARN_HEAPMAX as the preferred option of the two.
export YARN_TIMELINESERVER_HEAPSIZE={{apptimelineserver_heapsize}}

# Specify the JVM options to be used when starting the NodeManager.
# These options will be appended to the options specified as YARN_OPTS
# and therefore may override any similar flags set in YARN_OPTS
#export YARN_NODEMANAGER_OPTS=

# so that filenames w/ spaces are handled correctly in loops below
IFS=


# default log directory and file
if [ "$YARN_LOG_DIR" = "" ]; then
  YARN_LOG_DIR="$HADOOP_YARN_HOME/logs"
fi
if [ "$YARN_LOGFILE" = "" ]; then
  YARN_LOGFILE="yarn.log"
fi

# default policy file for service-level authorization
if [ "$YARN_POLICYFILE" = "" ]; then
  YARN_POLICYFILE="hadoop-policy.xml"
fi

# restore ordinary behaviour
unset IFS


YARN_OPTS="$YARN_OPTS -Dhadoop.log.dir=$YARN_LOG_DIR"
YARN_OPTS="$YARN_OPTS -Dyarn.log.dir=$YARN_LOG_DIR"
YARN_OPTS="$YARN_OPTS -Dhadoop.log.file=$YARN_LOGFILE"
YARN_OPTS="$YARN_OPTS -Dyarn.log.file=$YARN_LOGFILE"
YARN_OPTS="$YARN_OPTS -Dyarn.home.dir=$YARN_COMMON_HOME"
YARN_OPTS="$YARN_OPTS -Dyarn.id.str=$YARN_IDENT_STRING"
YARN_OPTS="$YARN_OPTS -Dhadoop.root.logger=${YARN_ROOT_LOGGER:-INFO,console}"
YARN_OPTS="$YARN_OPTS -Dyarn.root.logger=${YARN_ROOT_LOGGER:-INFO,console}"
export YARN_NODEMANAGER_OPTS="$YARN_NODEMANAGER_OPTS -Dnm.audit.logger=INFO,NMAUDIT"
export YARN_RESOURCEMANAGER_OPTS="$YARN_RESOURCEMANAGER_OPTS -Drm.audit.logger=INFO,RMAUDIT"
if [ "x$JAVA_LIBRARY_PATH" != "x" ]; then
  YARN_OPTS="$YARN_OPTS -Djava.library.path=$JAVA_LIBRARY_PATH"
fi
YARN_OPTS="$YARN_OPTS -Dyarn.policy.file=$YARN_POLICYFILE"
YARN_OPTS="$YARN_OPTS -Djava.io.tmpdir={{hadoop_java_io_tmpdir}}"", 
            "service_archive_root_dir": "{"DEFAULT":"hadoop-2.7.3", "METRICS_HADOOP_SINK":"ambari-metrics-hadoop-sink-2.5.1.0.0"}", 
            "service_check.queue.name": "default", 
            "min_user_id": "1000", 
            "yarn_cgroups_enabled": "false", 
            "yarn_user": "yarn", 
            "resourcemanager_heapsize": "1024", 
            "apptimelineserver_heapsize": "1024", 
            "yarn_data_dir": "/usr/ndp/data/yarn"
      }
    }
  }
}"
curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d "[{"Clusters":{
  "desired_config":[{
    "type":"yarn-env",
    "tag":"version2",
    "service_config_version_note":"New config version"}]}}]" "http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1"

基于API升级集群操作：

curl -H "X-Requested-By: ambari" -X POST -u admin:admin  http://hzadg-mammut-platform1.server.163.org:8080/api/v1/clusters/cluster1/upgrades -d @upgrade.json

> {"Upgrade":{"repository_version":"3.3.1.0.1","upgrade_type":"NON_ROLLING", "direction":"UPGRADE", "skip_prerequisite_checks":"true", "from_version":"3.2.1", "to_version":"3.3.1", "cluster_name":"cluster1"}}

AmbariServer操作相关

备份AmbariServer配置:

cd /var/lib/ambari-server/resources/scripts && python upgradeHelper.py –user=admin –password=admin –clustername=cluster2 –hostname=hzadg-mammut-platform5.server.163.org backup-configs

开启重启时包替换功能：

/var/lib/ambari-server/resources/scripts/configs.py –user admin –password admin –action set –host hzadg-mammut-platform1.server.163.org –cluster cluster1 –config-type cluster-env –key reinstall_component_package –value true

Ambari数据库相关操作

repo版本表：

select * from repo_version;

当前集群版本表：

update cluster_version set state=’CURRENT’ where id=1;

声明：该文观点仅代表作者本人，入门客AI创业平台信息发布平台仅提供信息存储空间服务，如有疑问请联系rumenke@qq.com。
  • 上一篇：没有了
  • 下一篇：没有了
未上传头像

copyright © 2008-2019 入门客AI创业平台 版权所有 备案号：湘ICP备2023012770号