Powered By Blogger

Wednesday, April 24, 2019

MapReduce materials


This has good explanation for java MR

This has good info on MR related to combiner,partitioner


 Mapper

Keyin, valuein, keyoutm , valueoutm

Reducer

Keyinm, valueinm, keyout , valueout


keyoutm  will be passed to reducer as Keyinm
Valueoutm will be passed to reducer as valueinm


MRUnit can be used for doing MR testing.







mapreduce-job-execution-flow-1-1024x492-1.jpg (1200×628)






Wednesday, April 3, 2019

Chef

Chef is configuration management tool for the hardware. It will be used to install softwarres, create folders with permissions, for doing patching and to open the ports.


Chef will have the workstation where the cookbook is developed having details about what to do when it gets executed on the node.


knife is the utility used to talk with chef server. It can be used to see the existing cookbooks, pushing cookbook to server, pushing cookbook to node etc
Knife needs to be configured with the chef server details.

Chef Server : it stores cookbooks , roles and users

Node : The node where the cookbook instructions will be executed.
sudo chef-client
is the command to be used to get the latest cookbooks from server.
Node will have the chef agent installed which will keep on talking to server.


https://www.youtube.com/watch?v=2BBC6sb4FQ0

https://www.youtube.com/watch?v=jlwGcgFfcnU