Databricks: Starting Journey

I just fired up the serverless compute engine on my free-account Databricks Edition — and I’m officially stepping into the world of data analytics and big-data processing.

This is something I wanted to do for a long time. With my background and the career shift I’m aiming for, I believe Databricks will help me level up my skills as a data engineer.

I’m excited and a little nervous. I plan to learn the basics first — creating a workspace, writing first notebooks, and experimenting with data pipelines. Over the coming days, I’ll post what I learn: mistakes, breakthroughs, and tips that help me (maybe they’ll help you too).

Monkey on the terrace

 https://www.youtube.com/shorts/mArXGk1qtCw

Long time

 Will start posts again

A Salesman's Tale.

 A Salesman's Tale.

The Salesman started his Friday work schedule with a light breakfast of sugar-bread. He tied his neck tie with precision, polished his shoes and helped himself with a coffee. His bag was ready with 10 encyclopedia's which he had to sell by End of the Day to meet his Friday Target else he would lose his weekly perk. He wrapped the bag around his shoulder took the bike keys from the holder.

Soon he was on his splendor wading his way to find customers. He would either ring or knock the doors and then use all his salesmanship skills to convince the customers to buy the books. This time for his surprise the books went off very fast and soon he was left with the last Tenth Book.

Strangely no one seemed to be ready to buy the book. What's the difference? He thought..It was the same encyclopedia as the rest.. He parked his bike below a tree and sat down near its roots. He took out his bottle of water and quenched his thirst wondering why he was not able to sell the book. He fished out his lunch box and ate some. But he was not happy and couldn't eat further  and thrust it back in to the bag. This book had to go somehow. 

He opened the book out of curiosity but found it no different than others. Customers were totally uninterested and some banged the doors at his face without even looking at the book. 

There was a beep and he could see his Boss's message in his mobile- 'all  books sold?'  He looked at the time in his mobile :It was already 3:00 PM. "No.. The last book is pending Sir, trying my best" He replied reluctantly. 

a little python web coding

 from flask import Flask

app = Flask(__name__)

print

@app.route('/')

def hello_world():

  return 'Hello, World'

@app.route('/myblog')

def my_blog():

  return 'my blog'

India Street in New Jersey

Lincoln Park in Snow [Feb 2021]

The Mechanical Horse

New York Skyline

 

Happy New Year!

 Wish all a happy and Prosperous New Year 2021!

Frozen Lake

 

Korukonda temple art a while ago

 

The Train Diary [with a difference]

I pushed my suitcase below the berth, carefully fastening the safety chain around it and placed my backpack on the upper berth as I settled down at my window seat, S3 Sleeper coach, Seshadri Express. Thankfully I had a confirmed ticket, which reserved several days in advance. It was already dark outside and could see my co-passengers taking out their dinner pouches, gearing for early supper. I will have late, I decided.

The train sped forward and the gentle breeze from the open window refreshed the grueling humid weather.

I looked at my watch, it was 8:30 PM now half hour since the train had started, I pulled out a short-stories book from bag and started reading it to ease the time.

"Tickets Please" The TTEs voice echoed its way into the compartment. He was a tall, bespectacled man donning a black suit, and a silver badge on the chest with the letters "TTE", with a bunch of papers in his hand. My co-passengers started showing their tickets one by one, he checked and noted in his papers. It was my turn, I showed my ticket. "ID please"

He said. I showed by "PAN" card, he nodded after examining it intently, satisfied and made a mark in his sheet. He then marched off saying "All good, Enjoy the journey!"

I again got absorbed into my book. 

"Tickets please!" The TTE was back again! 

AWS Certified Cloud Practitioner

Late Night Study

 Ravi had the habit of late-night study, he had his table and chair arranged in front of his bedroom window and would sit comfortably studying until he felt really sleepy. Then he would close

his books and sometimes doze off on the table only to wake up with a jerk and head to bed.

He also had the habit of taking coffee shots to ensure he stayed as late as possible.

It was Friday night. He started off as usual at his table, with his regular coffee shot, and started seriously taking notes. Then he noticed that it was getting foggy outside the window. The mist started dampening the panes.

He shifted back to his note-taking when suddenly his eyes fell on something animated in the dissolving fog, a lady! Clad in white with flowing matted hair and a lit candle in her hand. Was he dreaming?

He looked at his notes and could see that they were updated. He again looked out of the window, now he could see her silhouette slowly moving across the mist. 

He rubbed his eyes, and when he looked out...

SHE WAS RIGHT IN FRONT OF THE WINDOW!!

Develop Focus and Concentration in 15 minutes by Natural stream sounds

New York City Street Artist

The Thawing Triage Part 4

Baldev continued, "There has been a multi-disciplinary change in the organizational hierarchy which lead to a top-down approach of analyzing incidents rather than the bottom-up methodology and we are still in terms of streamlining this newer approach wherein Incidents are escalated and de-escalated as per the standard procedure which is still under de facto construction, which means that the intrinsic modus operandi has been either updated or upgraded, We will channelize the necessary forces to debug and detest the same."

"Yes you are right" said Cicily ,"The whole operational procedure needs to be re-visited so that the business impact is mitigated as quickly as possible as we are on the verge against the race of time"

What are they talking about?  Wondered San .

Meanwhile the Infrastructure guy confirmed on the channel that there were no issues observed within his workspace.

There are a beep sound. "Guy this is Challam (VenkataChallam) , Program Director for ITRADE support. I was pulled in to the call, Can someone brief me the issue and the steps being taken to resolve the issue with the Root Cause Analysis?"

Baldev Reiterated the issue.

"Do we know the clear impact? Have we downgraded the application? Do we know how much head room we have to mitigate the impact??" Rumbled Challam.

Baldev briefed him.

why dont they just record and play the issue? pondered San still debugging the issue.

 "Hands on deck guys hands on deck!" Shouted Challam. "We need to mobilize our forces to channelize our energies into issue resolution. Do what is to be done and try to expedite the issue, I need to get onto another call will be right back" And he disappeared.

Thank god, My ears are at ease thought San.

The Thawing Triage : Part 3

San dialed into the Triage and announced his name and team name, within seconds all the other relevant teams started dialing in one by one, whenever some one dialed in there was a beep sound, followed by their announcement.
Mr Jenkins made his statement : Seems every one necessary has dialled in, please keep the Bridge updated with information and if any help required done hesitate to contact me.

 ******** Bridge Opened for ITrade Application Outage **********

Bang on!!

The bridge started the operation,

Can someone please brief the issue?? I understand there is an outage in Itrade Application but don't have the specifics. Said Mr Benedict from the Infrastructure Team,

"We received alerts and then we started investigating the issue.. it seems.. "

"It was I who notified the Support on the incident" Suddenly Cicily screamed cutting sort of San's explanation.  Cicily was a business user who unknown to San was also called as High Pitch. She was known to scream and batter her way in calls. "Only after I called was the issue looked into!!"

"Guys Guys Guys!!" Yelled another voice "What is this I am hearing, who is from Support? Why wasnt this looked upon proactively?? We should have notified the stakeholders before they coming to s...  I can see a huge gap here.. This is Baldev here" Baldev was the Project Manager for ITrade.

San was perturbed, "What's happening?" He thought, people are more focused on rants than issue resolution. And the gap he was talking about might be in milli seconds!!

San stepped forward. "I am in the shift currently, I saw the alert and promptly followed the protocols.."

"This is not about just following the protocols, we need to communicate fastidiously to the stakeholders. Can you send notification asap!!" Thundered Baldev.

"I already sent .. "

"Send another with the current status that bridge is open and the appropriate steps are being taken to mitigate the Risk associated with the Outage ASAP!"

San head started aching, He looked at Bakhsi who was eating popcorn now sitting in his room. Lucky fellow he thought.  He pinged him.

San: Bakshi, i might need some help in driving the call, everybody is banging me left and right

Bakshi: Sorry yaar, I am really busy, I am working on business analytics to forecast which needs to be delivered within 6 months, really busy..

Six months?? You said as if it were 6 minutes!! thought San.
There is no use, San understood, he had to drive the call himself.

Baldev continued his speech. "Cicily I am extremely sorry that this had to come to you to gain traction, we will note this in our AGM Dashboard and will be streamlining the process with Multitude Pipeline methodology. Let's focus on the resolution currently and we will take up the rest with our Post Incident Review call."

San started reviewing the back end scripts now. He was trying to understand the issue as why the process went down in the first place, however he could see the script was more than 1K lines long without any proper documentation.

Wait Three Seconds!

An interesting piece of code from Java Script : Understanding the Weird Parts

//long running function

function waitthreeseconds(){

    var ms = 3000 + new Date().getTime();

    while (new Date() < ms){

        

    }

    console.log("Function Finished")

    }

waitthreeseconds();