Ambari User Views: Tech Preview



Ambari User Views:

Tech Preview







 



WelcometoHortonworksAmbariUserViewsTechnicalPreview.ThisTechnicalPreview

providesearlyaccesstoupcomingfeatures,lettingyoutestandreviewduringthe

developmentprocess.Thesefeaturesareconsideredunderdevelopmentandarenot

intendedforuseinyourproductionsystemsandarenotsupportedbyHortonworksbutyour

feedbackisgreatlyappreciated.HavefunandpleasesendfeedbackontheHortonworks

Communityforumshttp://hortonworks.com/community/forums/forum/ambari



 



TableofContents

Introduction

SystemRequirements

ViewPackageDownloads

InstallAmbariandHDP

DeploytheViews

ConfiguretheFilesView

ConfiguringHDFS

CreatingaViewInstance

ConfigurethePigView

ConfiguringHDFS

ConfiguringWebHCat

CreatingaViewInstance

ConfiguretheHiveView

ConfiguringHDFS

CreatingaViewInstance

ConfiguretheCapacitySchedulerView

CreatingaViewInstance

ConfiguretheTezView

CreatingaViewInstance

ConfiguringAmbariUserViewsforKerberos

CreateAmbariServerProxyUserPrincipalandKeytab

SetupAmbariServerforKerberoswithProxyUser

ConfigureClusterServicesforAmbariProxyUser

ConfigureAmbariUserViewsforKerberosAuthentication

FilesView

HiveView

PigView

CapacitySchedulerView

LearnMoreAboutViews

Troubleshooting

Issue:WhenexecutingaHivequeryfromtheView:

Issue:WhenloadingtheCapacitySchedulerView

Solution:ChecktheviewconfigurationandconfirmtheAmbariClusterURLand

credentialsprovidedarecorrectforyourcluster.

Appendix:UsersandPermissions

SetupAmbariandHDPUsers

ManageViewAccessPermissions



Introduction

TheHortonworksAmbariUserViewsTechnicalPreviewprovidesasetofviewstotryout

withAmbari.ForbackgroundonAmbari,theAmbariViewsFramework,andandthecurrent

ViewsinthisTechnicalPreviewreviewthefollowing:



ApacheAmbari

Operationalplatformforprovisioning,managingandmonitoring

ApacheHadoopclusters.AmbariexposesarobustsetofREST

APIsandarichWebinterfaceforclustermanagement.

AmbariViews

framework

InitiallymadeavailableaspartofAmbari1.7.0,theAmbariViews

frameworkoffersasystematicwaytopluginUIcapabilitiesto

surfacecustomvisualization,managementandmonitoring

featuresinAmbariWeb.

AmbariUserViews

SpecificextensionsoftheAmbariWebinterfacevia“userviews”.

TheAmbariUserViewsTechnicalPreviewusesthisthe

underlyingAmbariViewsframeworktodeliverviewsforPig,Tez,

Hive,FilesandCapacityScheduler.

PigView

ProvidesawaytoauthorandexecutePigscripts.

SeeConfigurethePigView.

HiveView

Exposesawaytofind,authorandexecuteHivequeries.

SeeConfiguretheHiveView.

TezView

IntroducedaspartofAmbari2.0andHDP2.2.4,itallowsyouto

debugHiveQueriesandPigscripts.Ifconfigured,itcanalsobe

accessedfromtheHiveView.

SeeConfiguretheTezView.

FilesView

AllowsyoubrowsetheHDFSfilesystem.

SeeConfiguretheFilesView.

CapacityScheduler

View

ProvidesavisualwaytoconfigureYARNcapacityscheduler

queuecapacity.



SeeConfiguretheCapacitySchedulerView.



System Requirements

ThefollowingliststhesystemsrequirementsfortheTechPreview:

● Ambari2.0+HDP2.2.4Cluster

● InstallofHDFS,YARN+MapReduce2,Tez,Pig,Hive,ZooKeeper,AmbariMetrics

View Package Downloads

ThefollowingliststhesoftwaredownloadsfortheAmbariUserViewsTechPreview:

Item

DownloadURL

Ambari2.0

RefertotheAmbari2.0documentationforinformationoninstalling

andconfiguringAmbari.

http://docs.hortonworks.com/HDPDocuments/Ambari2.0.0.0/index.ht

FilesView

http://publicrepo1.hortonworks.com/HDPLABS/Projects/Views/tp1/fil

es0.1.0tp1.jar

HiveView

http://publicrepo1.hortonworks.com/HDPLABS/Projects/Views/tp1/h

ive0.2.0tp1.jar

PigView

http://publicrepo1.hortonworks.com/HDPLABS/Projects/Views/tp1/p

ig0.1.0tp1.jar

Capacity

SchedulerView

http://publicrepo1.hortonworks.com/HDPLABS/Projects/Views/tp1/c

apacityscheduler0.3.0tp1.jar

TezView

AlreadypackagedinAmbari2.0.

Install Ambari and HDP



1. DownloadandinstalltheAmbari2.0release.

http://docs.hortonworks.com/HDPDocuments/Ambari2.0.0.0/index.html

2. LaunchtheAmbariClusterInstallWizardandinstallanHDP2.2.4cluster.Besure

toincludethefollowingservicesatminimum:HDFS,YARN+MapReduce2,Tez,

Hive,Pig,ZooKeeperandAmbariMetrics.

3. ProceedtoDeploytheViews.



Deploy the Views



1. AfterinstallingAmbariandHDP,downloadtheviewpackagesusingtheURLsabove.



2. CopytheviewpackagestotheAmbariServerintheviewsdirectory.Forexample:

cp*tp1.jar/var/lib/ambariserver/resources/views/



3. RestarttheAmbariServertodeploytheviews.

ambariserverrestart



4. OncetheviewsaredeployedinAmbariServer,browsetotheAmbariWebUIand

http://ambari.server:8080



5. GototheAmbariAdministrationinterfacebyselectingManageAmbari:



6. Followconfigurationstepsbelowspecifictoeachview.



7. RefertotheAmbariAdministrationGuideformoreinformationonManagingViews.



Configure the Files View



TheFilesViewprovidesaconvenientwaytoaccessHDFSthroughawebbasedinterface.

Configuring HDFS



YouneedtosetupanHDFSproxyuserfortheAmbaridaemonaccount.Forexample,if

ambariserverdaemonisrunningasroot,yousetupaproxyuserforrootincoresite.xmlby

clickingHDFS>Configs>Customcoresite>AddProperty



:



hadoop.proxyuser.root.groups=*

hadoop.proxyuser.root.hosts=*



RestarttherequiredcomponentsasindicatedbyAmbari.Ifyouwanttoaddmultipleviews,

youcanrestartservicesonceaftermakingchangesforallviews.

Creating a View Instance



1. BrowsetotheAmbariAdministrationinterface.

2. ClickViews,expandtheFilesviewandclickCreateInstance.

3. Enterthefollowinginformation:



Property

Value

InstanceName

FILES_1

DisplayName

MyFiles

Description

BrowseHDFSfilesanddirectories

WebHDFSFileSystemURI

webhdfs://<HDFS>configs>Advancedhdfssite>

dfs.namenode.httpaddress



>

E.g.

webhdfs://c6401.ambari.apache.org:50070

WebHDFSUsername

${username}



4. ClickSave,GivePermissionstotheappropriateusersandgroups,andgotheview

theinstance.



Configure the Pig View



ThePigViewprovidesawebbasedinterfacetocompose,editandsubmitqueries,download

results,viewlogsandviewhistoryofjobsubmissions.



Configuring HDFS



YouneedtosetupanHDFSproxyuserfortheAmbaridaemonaccount.Forexample,if

ambariserverdaemonisrunningasroot,yousetupaproxyuserforrootincoresiteby

addingandchangingpropertiesinHDFS>Configs>Customcoresite



:



hadoop.proxyuser.root.groups=*

hadoop.proxyuser.root.hosts=*



YouneedtosetupanHDFSproxyuserforWebHCat.Forexample,ifyourWebHCatServer

isrunningashcat,yousetupaproxyuserforhcatincoresite:



hadoop.proxyuser.hcat.groups=*

hadoop.proxyuser.hcat.hosts=*

Configuring WebHCat



YouneedtosetupaWebHCatproxyuserfortheAmbaridaemonaccount.SelectHive>

Configs>Customwebhcatsite>AddProperty



.Forexample,ifambariserverdaemonis

runningasroot,yousetupaproxyuserforrootinwebhcatsite:



webhcat.proxyuser.root.groups=*

webhcat.proxyuser.root.hosts=*

Creating a View Instance



1. BrowsetotheAmbariAdministrationinterface.

2. ClickViews,expandthePigviewandclickCreateInstance.

3. Enterthefollowinginformation:



Property

Value

InstanceName

PIG_1

DisplayName

MyPig

Description

SaveandexecutePigscripts

WebHDFSFileSystemURI

webhdfs://<HDFS>configs>Advancedhdfssite>

dfs.namenode.httpaddress



>

E.g.

webhdfs://c6401.ambari.apache.org:50070

WebHCatURL

FromHive>Configs

http://<WebHCatServerhost>:<templeton.port>/templeton/v1



E.g

http://c6401.ambari.apache.org:50111/templeton/v1

JobsHDFSDirectory

/user/${username}/pig/jobs

ScriptsHDFSDirectory

/user/${username}/pig/scripts



4. ClickSave,GivePermissionstotheappropriateusersandgroups,andgotheview

theinstance.



Configure the Hive View

Configuring HDFS



YouneedtosetupanHDFSproxyuserfortheAmbaridaemonaccount.Forexample,if

ambariserverdaemonisrunningasroot,yousetupaproxyuserforrootincoresiteby

clickingHDFS>Configs>Customcoresite>AddProperty



:



hadoop.proxyuser.root.groups=*

hadoop.proxyuser.root.hosts=*



Inaddition,itisnecessarytosetupHDFSusersforallAmbariusersthatyouplanonusing;

otherwiseyouwillreceivepermissionerrorsonjobsubmissionsandfilesaveoperations.

Creating a View Instance



1. BrowsetotheAmbariAdministrationinterface.

2. ClickViews,expandtheHiveviewandclickCreateInstance.

3. Enterthefollowinginformation:



Property

Value

InstanceName

HIVE_1

DisplayName

MyHive

Description

SaveandexecuteHiveSQLscripts

WebHDFSFileSystemURI

webhdfs://<HDFS>configs>Advancedhdfssite>

dfs.namenode.httpaddress



>

E.g.



webhdfs://c6401.ambari.apache.org:50070

WebHCatURL

FromHive>Configs

http://<WebHCatServerhost>:<templeton.port>/templeton/v1

E.g

http://c6401.ambari.apache.org:50111/templeton/v1

JobsHDFSDirectory

/user/${username}/hive/jobs

ScriptsHDFSDirectory

/user/${username}/hive/scripts

HiveServer2Host

ClickHive>Summary>HiveServer2



andnotethehostname

E.g.

c6401.ambari.apache.org

HiveServer2Port

<Hive>Configs>HiveServerPort>

E.g.

10000

HiveAuth

auth=NONE;user=${username}

yarn.ats.url*

http://<YARN>Configs>

yarn.timelineservice.webapp.address>

E.g.

http://c6401.ambari.apache.org:8188



4. ClickSave.GivePermissionstotheappropriateusersandgroups,andgotheview

theinstance.



!

TheTezViewintegrateswiththeHiveView.PleaseinstalltheTezView

whenyouinstalltheHiveView.

Configure the Capacity Scheduler View



TheCapacitySchedulerViewprovidesaUItomanagetheYARNcapacityscheduler

queueseliminatingcumbersomeeditingoftheunderlyingXMLfiletypicallyusedtoconfigure

thecapacityscheduler.TheviewusestheAmbariRESTAPItoreadandmodifythecapacity

schedulerconfigurationfile.

Creating a View Instance



1. BrowsetotheAmbariAdministrationinterface.



2. ClickViews,expandtheCapacitySchedulerviewandclickCreateInstance.

3. Enterthefollowinginformation:



Property

Value

InstanceName

CS_1

DisplayName

MyQueueManager

Description

BrowseandmanageYARNCapacitySchedulerqueues

AmbariClusterURL

http://<AmbariHost>:8080/api/v1/clusters/<clusterName>

E.g.

http://c6401.ambari.apache.org:8080/api/v1/clusters/foo

OperatorUsername

<operatorUsername>

OperatorPassword

<operatorPassword>



4. ClickSave,GivePermissionstotheappropriateusersandgotheviewtheinstance.



Configure the Tez View

TezisanexecutionenginethatcanbeusedforHiveandPig.TheTezViewprovidesthe

abilitytounderstandanddebugTeztaskexecution,duringandafterexecution.



!

TheTezViewisincludedwithAmbari2.0.Separatedownloadisnotrequired

asitisGAandcomespredeployedinAmbari2.0

Creating a View Instance

1. BrowsetotheAmbariAdministrationinterface.

2. ClickViews,expandtheTezviewandclickCreateInstance.

3. Enterthefollowinginformation:



Property

Value

InstanceName

TEZ_1

DisplayName

MyTez

Description

Find,understandanddebugTezjobs

YARNTimelineServerURL*

http://<YARN>Configs>yarn.timelineservice.webapp.address>

E.g.

http://c6401.ambari.apache.org:8188

YARNResourceManagerURL*

http://<YARN>Configs>yarn.resourcemanager.webapp.address>

E.g.

http://c6401.ambari.apache.org:8088



4. ClickSave.GivePermissionstotheappropriateusersandgroups,andgotheview

theinstance.



Configuring Ambari User Views with a Secure Cluster



ThisdocumentdescribeshowtoconfiguretheAmbariUserViewstoworkwitha

Kerberosenabledcluster.



● CreateAmbariServerProxyUserPrincipalandKeytab

● SetupAmbariServerforKerberoswithProxyUser

● ConfigureClusterServicesforAmbariProxyUser

● ConfigureAmbariUserViewsforKerberosAuthentication



Create Ambari Server Proxy User Principal and Keytab

UsingtheKerberoskadminutility[inthecaseofMITKDC],createtheAmbariService

principalthatwillactastheProxyUser.Thenameusedfortheprincipalisarbitrary.Forthis

example,wewilluse:

ambariuser/ambariserver[email protected]



!

TheHiveViewrequiresthattheprincipalnameiscomprisedofthreeparts:the

primaryuser,theinstanceandtherealm(separatedby“/”and“@”).Forexample:

primary/instance@realm.



Asamplesessionwithkadmincouldlookasfollows:



root@kdc1~]#kadmin.local

Authenticatingasprincipalroot/[email protected]withpassword.

kadmin.local:add_principalpwhadoopambariuser/ambariserverhost

WARNING:nopolicyspecifiedforambariuser/ambariserver[email protected];

defaultingtonopolicy

Principal"ambariuser/ambariserver[email protected]"created.



Next,generateakeytabforthenewlycreatedprincipal:



kadmin.local:xstnorandkeykambariuser.keytab

ambariuser/ambariserver[email protected]