Invictus

Invictus” by William Ernest Henley

Out of the night that covers me,
Black as the Pit from pole to pole,
I thank whatever gods may be
For my unconquerable soul.

In the fell clutch of circumstance
I have not winced nor cried aloud.
Under the bludgeonings of chance
My head is bloody, but unbowed.

Beyond this place of wrath and tears
Looms but the horror of the shade,
And yet the menace of the years
Finds, and shall find me, unafraid.

It matters not how strait the gate,
How charged with punishments the scroll,
I am the master of my fate;
I am the captain of my soul

Share: herbert, frank – children of dune.htm

Atrocity is recognized as such by victim and perpetrator alike, by all who learn about it at whatever remove. Atrocity has no excuses, no mitigating argument. Atrocity never balances or rectifies the past. Atrocity merely arms the future for more atrocity. It is self-perpetuating upon itself — a barbarous form of incest. Whoever commits atrocity also commits those future atrocities thus bred.

Share: herbert, frank – dune messiah.htm

Constitutions become the ultimate tyranny,” Paul said. “They’re organized power on such a scale as to be overwhelming. The constitution is social power mobilized and it has no conscience. It can crush the highest and the lowest, removing all dignity and individuality. It has an unstable balance point and no limitations. I, however, have limitations. In my desire to provide an ultimate protection for my people, I forbid a constitution. Order in Council, this date, etcetera, etcetera.”

How (not?) to be decision mechanic:

Disclaimers:

 *-- This is a summary of all my experience(working in various roles in various startups/small companies, not specific to any one experience)
 * -- I've argued with founders being a early engineer, I've argued with line-managers in a >1000 person organization, but have been successful convincing the decision-makers less than 50% of the decisions. So the end of the day, [skin-in-the-game](https://www.amazon.com/Skin-Game-Hidden-Asymmetries-Daily/dp/042528462X), might be a a much bigger impact on persuasion ability than anything below, but what's below can't hurt.
* -- This was inspired by the types of data scientist post [this](https://medium.com/hackernoon/top-10-roles-for-your-data-science-team-e7f05d90d961) but I've always thought of myself as a [generalist](https://www.merriam-webster.com/dictionary/generalist)

To do:

* -- Follow the [5W-1H](https://www.isixsigma.com/implementation/basics/using-five-ws-and-one-h-approach-six-sigma/)
* -- Ask about the if-else decision-action tree the decision maker(s) with most
  "skin-in-the-game" will have to use.*
* -- Proposing any potential/possible alternatives to the decision-action tree above is the
  job of an analyst, if you're playing that role make it clear, it is a role-playing you're
  doing rather than primary responsiblity tied to your performance evaluations.
* -- When? well, leave it completely  to the Decision Maker and suggestions from the
  statisticians and data scientists(based on forecasting models perhaps??) Try not to play
  too many roles, you're likely to make more mistakes that way.

* -- Where? This is something you can try your hand, but i highly recommend testing the
  waters well, ideally, this would be decided, by a combination of data visualization
  experts/analysts + decision maker + statisticians.
* -- How? Leave this completely to the Data Scientists and S/W Architects and others with
  stake in the outcomes.
* -- Why?? Now this is your complete responsibility, Make sure you don't make any mistakes
  on this area. The biggest advice I can give on avoiding mistakes on this is that don't
  take your eyes of the [0th and 1st virtues of rationality](https://yudkowsky.net/rational/virtues).
  Make you balance both your curiousity and focus, The rest of the virtues are important
  too, but personally I find mistakes originating from failure to focus on balancing 0th and
  1st.

That’s all folks, Hope you’ve a good set of adventures.

• — Some of them may not like to think about this before or commit to a decision tree, you’ll have
  to ask, but might have to settle for a “gut instinct” answers for a few scenarios you can imagine.

Some doodles I found when cleaning up old notebooks.

Technical Debt in ML models

Technical Debts are also there in ML:

Complex models erode boundaries

* -- Entanglement of features and feature distributions
* -- Correction cascades creating cascade chains of models and dependency hell
* -- Undeclared consumers for the model predictions

Data dependencies are costlier than code dependencies

* -- Unstable Data Dependencies unstable input data or signals or predictions from a previous model..(For ex: in [speech to text](https://github.com/kaldi-asr/kaldi), the syllables is a prediction and signal/input to the word-language model)
* -- Underutilized Data Dependencies   (Creep in via Legacy Features, Bundled Features,
  Correlated Features etc)
* -- Static analysis of data dependencies can help mitigate these issues to some extent

Feedback loops

* -- Direct Feedback loops(In speech to text ,it can come from changes in languages and
  pronounciation)
* -- Hidden Feedback loops (These can come from not understanding the business use-case as
  explicitly as possible or other things like change in the nature of the use-case and
  demand itself. For ex: user expectations changing after getting used to the tool)

ML-system anti-patterns:

* -- Glue code -- in general, things like cleaning code, connecting model prediction, and
  business use-case etc.
* -- Pipeline Jungles  Huge mess of pre-processing of audio files.. different formats,
  different language and accent detection(this can also be cascaded models) etc..

* -- Dead code on Experimental codepaths: probably from a bunch of experimental models
  different NN architectures, different custom models etc..

* -- Abstraction Debt: No clear standard abstraction for ML models. (like RDBMS for
  database)

Common Smells:

* --  Plain-old-data type smells.. assume some data types but the input stream is
  changing...
* -- Multiple Language smell: this is programming language and how using multiple languages
  in a project cause multiple problems/issues at the interfaces.
* -- Prototype smell: The prototype is written and makes invalid assumptions. Even whatever
  validation that has been done for the prototype is not valid outside of the small audience
  this was tested on.

Configuration Debt:

* -- Wide range of configurable options from input data stream segregation/categorizations,
  model size and dependencies tuned to latency/thoroughput of the predictions, model choice,
  input features, data summarization methods, verification methods etc..
* -- If there's a lack of configuration management the system can become a black box
  impossible to debug and therefore improve. While these are similar to common software
  applications, these are doubly problematic in ML models as a lot of models are considered
  black-box by default and are already hard to reason about without these configuration
  issues.

Dealing With Changes in the external world.

* -- Fixed thresholds in Dynamic Systems:
* -- Monitoring and Testing for the model's failure limits (for ex: in case of a data
  outlier)
    Things to monitor: * -- Prediction Bias
               * -- Action Limits(say a trading algo relying on a model should
                 have limits)
               * -- Up-stream Producers (aka data pre-processing pipelines, for
                 ex: a moving window of 100 ticks/events may not be right for
                 different(higher) velocity of input data.)

Others:

* -- Data testing Debt
* -- Reproducibility Debt
* -- Process Management Debt
* -- Cultural Debt

Laptop purchase decision

So in the past, I’ve ranted about the “confusion marketing” in the laptop market. (see here).

So this time around after more than 5 years, when i had to buy a new laptop, I decide to apply some analytical ideas, i’ve picked up over this time working.

So I created this sheet, which helped me out.

Since i had written a series of posts in the past about different types of mean, i had created this blog post.
So I ended up buying this.

Yentl Syndrome: A Deadly Data Bias Against Women

Longreads

Caroline Criado Perez | An excerpt adapted from Invisible Women: Data Bias in a World Designed for Men | Harry N. Abrams | 22 minutes (5,929 words)

In the 1983 film Yentl, Barbra Streisand plays a young Jewish woman in Poland who pretends to be a man in order to receive an education. The film’s premise has made its way into medical lore as “Yentl syndrome,” which describes the phenomenon whereby women are misdiagnosed and poorly treated unless their symptoms or diseases conform to that of men. Sometimes, Yentl syndrome can prove fatal.

If I were to ask you to picture someone in the throes of a heart attack, you most likely would think of a man in his late middle age, possibly overweight, clutching at his heart in agony. That’s certainly what a Google image search offers up. You’re unlikely to think of a woman: heart disease is…

View original post 6,045 more words

Aruvi(2017) — review

Movie: Aruvi

Brilliance of screenplay irony:

• — The scene at the shooting set after the shooting, fights with that cheater, exploitative guy.
  The irony arises from the numbness/indifference of the actress, and the way she makes puppets out
  of the shooting crew, which for a change have to face real life-or-death drama rather than the
  made-up things they shoot. Not to mention the background bass.

• — The idealist wannabe director’s impractical dream story to direct.

• — The role reversal of the prima donna actress serving tea.

• — Ofcourse, the mock-tv show, of the actress.

Some things about the movie that left me cold though
* — The manner how the protagonist got HIV infection is a rather low probabilty and rare method(it had to go from the live blood of coconut seller to, any bleeding gums in the protagonist mouth or other bleeding wounds in the digestive tract), and the director had to reach for it to avoid the pre-marital sex taboo culture. I personally think it is a meaningless one to stick to. (Not that i suggest we let US style marketing and companies use dating and sex as a lure. Ironically, sticking to cultural excuses would drive some youth to that approach only. )

“Is it?  It really doesn’t feel like that’s the case, right now,” I answered.  “Home’s supposed to be a place I feel safe and secure.”