Quick Notes

Things that came on the way

Note on Distributed Computing

A distributed system is a collection of independent computers that appears to its users as a single coherent system. This paper argues that the objects in a distributed object oriented system form a single ontological class where all entities can be described by the specification of the set of interfaces of the objects and the semantics of operation is mistaken. This vision of unified objects for distributed systems is centered around the principles that

End-to-end Arguments in System Design

This paper presents the design principle regarding placement of functions in computer system design called “end-to-end argument”. The argument of the principle is that any application functions implemented at lower levels of a system may be redundant or of little value when compared with the cost of providing them at lower level. The paper articulates the argument through requirements and examples in distributed systems like reliable data transmission, encryption, duplicate message detection, message sequencing, detecting host crashes, delivery receipts etc.

Scale in Distributed Systems

This paper looks at scale and how it affects distributed systems including highlights of how how scale is addressed in existing systems. A system is said to be scalable if it can handle addition of resources and users without suffering noticeable loss in performance or increase in administrative complexity. Scale also affects the way users perceive the systems. For e.g. as the number of objects accessible grows it becomes increasingly difficult to locate the objects of interest.

Oozie Job to Schedule HBase Major Compaction

HBase performs time based major compaction and in-order to prevent this resource intensive process interfere performance sensitive applications, this can be disabled. Once disabled, in order to keep the HBase store files in optimal condition, application team need to schedule regular compaction. The following details the steps which need to be followed to schedule daily compaction through Oozie.