Life on the Poincaré Disk

Point at Infinity

Just at this time I left Caen, where I was then living, to go on a geological excursion under the auspices of the school of mines. The changes of travel made me forget my mathematical work. Having reached Coutances, we entered an omnibus to go some place or other. At the moment when I put my foot on the step the idea came to me, without anything in my former thoughts seeming to have paved the way for it, that the transformations I had used to define the Fuchsian functions were identical with those of  non-Euclidean geometry. I did not verify the idea; I should not have had time, as, upon taking my seat in the omnibus, I went on with a conversation already commenced, but I felt a perfect certainty. On my return to Caen, for conscience’ sake I verified the result at my leisure.

-Henri Poincaré, Science and…

View original post 2,107 more words


Math Classes Every College Should Teach

Math with Bad Drawings

Math 40: Trying to Visualize a Fourth Dimension. Syllabus includes Flatland, the Wikipedia page for “hypercube,” long hours of squinting, and self-inflicted head injuries.

Image (6).jpgMath 99: An Irritating Introduction to Proof. The term begins with five weeks of the professor responding to every question with, “But how do you knoooooooow?” If anyone is still enrolled at that point, we’ll have to wing it, since no one has ever lasted that long.

Image.jpgMath 101: Binary. An introductory study of the binary numeral system. Also listed as Math 5.

View original post 278 more words

Topological maps or topographic maps?

David Richeson: Division by Zero

While surfing the web the other day I read an article in which the author refers to a “topological map.” I think it is safe to say that he meant to write “topographic map.” This is an error I’ve seen many times before.

A topographic map is a map of a region that shows changes in elevation, usually with contour lines indicating different fixed elevations. This is a map that you would take on a hike.

A topological map is a continuous function between two topological spaces—not the same thing as a topographic map at all!

I thought for sure that there was no cartographic meaning for topological map. It turns out, however, that there is.

A topological map is a map that is only concerned with relative locations of features on the map, not on exact locations. A famous example is the graph that we use to…

View original post 95 more words

Python’s Weak Performance Matters

Meta Rabbit

Here is an argument I used to make, but now disagree with:

Just to add another perspective, I find many “performance” problems in
the real world can often be attributed to factors other than the raw
speed of the CPython interpreter. Yes, I’d love it if the interpreter
were faster, but in my experience a lot of other things dominate. At
least they do provide low hanging fruit to attack first.


But there’s something else that’s very important to consider, which
rarely comes up in these discussions, and that’s the developer’s
productivity and programming experience.[…]

This is often undervalued, but shouldn’t be! Moore’s Law doesn’t apply
to humans, and you can’t effectively or cost efficiently scale up by
throwing more bodies at a project. Python is one of the best languages
(and ecosystems!) that make the development experience fun, high
quality, and very efficient.

(from Barry Warsaw)


View original post 705 more words

Seat Belts, Condoms and the Indian Equity Investor

The Eighty Twenty Investor

The Seat Belt mystery

Image result for seat belts

In early 1970s, when the use of seat belts were made mandatory in the US to improve driver safety, something strange happened.

Instead of road accident deaths coming down they actually went up!

While the regulators were perplexed by this phenomenon, an economist by the name Sam Peltzman came up with a controversial answer.

Image result for seat belts + peltzman

He argued that though the drivers had lower risks due the additional safety that a seat belt provides,  many drivers actually compensated for the additional safety by driving more recklessly (driving faster, not paying as much attention, etc.) under the comfort of the added safety.

“The safer they make the cars, the more risks the driver is willing to take”

This meant that bystanders – pedestrians, bicyclists etc – would receive no safety benefit from the seat belts but would rather suffer as a result of increased recklessness.

He termed this…

View original post 1,643 more words

Data Science: Structured thinking — a collection of guide.

Inspired by this. Read it first:

  1. Figure out the questions involved in the analytics project and decide which ones can be tackled
    separately, and which ones are intertwined with others, and which ones need to be answered first
    before tackling others. Then pick one.
    0.5 Layout the data requirements and hypothesis before looking at what data is available
  2. Actually Look at the data summary(dataframe.describe()) that includes mean, mode, std, and quartiles)
  3. Look for patterns in the summary. Think about what each of the values mean to your question? What
    do questions do they lead to? How do they modify your question?
  4. Figure out the ML problem use this.

  5. Go back to step 1 and 2 again and redo them with the ML problem .
  6. See if you have enough data (noise vs signal) or you need more samples or do you need more
    features. (see

First Model building time-split:
1.Descriptive analysis on the Data – 50% time
2.Data treatment (Missing value and outlier fixing) – 40% time
3.Data Modelling – 4% time
4.Estimation of performance – 6% time

Data Exploration steps:
Source Reference:
Below are the steps involved to understand, clean and prepare your data for building your predictive model:

    1.Variable Identification
    2.Univariate Analysis
    3.Bi-variate Analysis
    4.Missing values treatment
    5.Outlier treatment
    6.Variable transformation
    7.Variable creation

Missing Value Treatment:
    2.Mean/ Mode/ Median Imputation
    3.Prediction Model:
    4.KNN Imputation:
Outlier Treatment:
    1.Data Entry Errors:
    2. Measurement Error:
    3. Experimental Error:
    4. Intentional Outlier:
    5. Data Processing Error:
    6. Sampling error:
    7. Natural Outlier:

Exploring the ChestXray14 dataset: problems

Luke Oakden-Rayner

A couple of weeks ago, I mentioned I had some concerns about the ChestXray14 dataset. I said I would come back when I had more info, and since then I have been digging into the data. I’ve talked with Dr Summers via email a few times as well. Unfortunately, this exploration has only increased my concerns about the dataset.

WARNING: there are going to be lots of images today. If data and bandwidth are a problem for you, beware. Also, this piece is about 5000 words long. Sorry :)
DISCLAIMER: Since some people are interpreting this wrongly, I do not think this piece in any way reflects broader problems in medical deep learning, or suggests that claims of human performance are impossible. I’ve made a claim like that myself, in recent work. These results are specific to this dataset, and represent challenges we face with medical data. Challenges…

View original post 5,094 more words

Word_2_vector.. (aka word embeddings)

Word 2 vector:

  • word 2 vector is a way to take a big set of text and convert into a matrix with a word at
    each row.
  • It is a shallow neural-network(2 layers)
  • Two options/training methods (

CBOW(Continuous-bag-of-words assumption)

  • — a text is represented as the bag(multiset) of its words
  • — disregards grammar
  • — disregards word order but keeps multiplicity
  • — Also used in computer vision

skip-gram() — it is a generalization

of n-grams(which is basically a markov chain model, with (n-1)-order)
* — It is a n-1 order markov model
* — Used in Protein sequencing, DNA Sequencing, Computational
linguistics(character and word)
* — Models sequences, using the statistical properties of n-grams
* — predicts x_i based on x_(i-(n-1)), ....,x_(i-1) .
* — in language modeling independence assumptions are made so that each
word depends only on n-1 previous words.(or characters in case of
character level modeling)
* — The probability of a word conditional on previous n-1 words follows a
Categorical Distribution
* — In practice, the probability distributions are smoothed by assigning non-zero probabilities to unseen words or n-grams.

Bias-vs-Variance Tradeoff:

  • — Finding the right ‘n’ for a model is based on the Bias Vs Variance tradeoff we’re wiling to make

Smoothing Techniques:

  • — Problems of balance weight between infrequent n-grams.
  • — Unseen n-grams by default get 0.0 without smoothing.
  • — Use pseudocounts for unseen n-grams.(generally motivated by
    bayesian reasoning on the sub n-grams, for n < original n)

  • — Skip grams also allow the possibility of skipping. So a 1-skip bi(2-)gram would create bigrams while skipping the second word in a three sequence.

  • — Could be useful for languages with less strict subject-verb-object order than English.

Alternative link

  • Depends on Distributional Hypothesis
  • Vector representations of words called “word embeddings”
  • Basic motivation is that compared to audio, visual domains, the word/text domain treats
    them as discrete symbols, encoding them as sparse dataset. Vector based representation
    works around these issues.
  • Also called as vector Space models
  • Two ways of training: a, CBOW(Continuous-Bag-Of-Words) model predicts target words, given
    a group of words, b, skip-gram is ulta. aka predicts group of words from a given word.

  • Trained using the Maximum Likelihood model

  • Ideally, Maximizes probability of next word given the previous ‘h’ words in terms of a softmax function
  • However, calculating the softmax values requires computing and normalizing each probability using score for all the other words in context at every step.
  • Therefore a logistic regression aka binary classification objective functionis used.
  • The way this is achieved is called negative sampling

The Cauvery Water debate — opinions

This was inspired by the controversy around cauvery water debate in mid-september 2016.

Before we begin, I’ll set down my biases and priors and  assumptions:

  1.    I’m from TN, living in bangalore for about 10 years.
  2.    I’m unaware of the actual rain level, agricultural needs, ecological needs and others.
  3.   I’m not going to propose a verdict as much as a process/method to deal with the conflicts that don’t depend on politicians or supreme court.
  4.  I’ve travelled in TN, to most parts in my youth, and trekked to most parts of Karnataka(speaking broken kannada) in the last 10 years,(obviously not as much as TN) and have a grasp on the cultural/mental attitudes in general.


One reason I’m ruling out political solution is because we live in a representational democracy.  The way the incentives are in that setup are for the politicians to do what gets them the most votes from the biggest part of their demographics. Trying to expect them to talk to politicians from other state and  come to a compromise is hard because on top of representational democracy, we have a  multi-party system.  Which means, there’s scope for local parties to not care about the interest of the other state parties and people. I’ve seen a few national parties taking contradictory stances, based on which state’s division they are making statements from.  In addition to this incentives, this is a situation with  prisoner’s dilemma type dynamics(i.e: to say, if one agent stops co-operating and defects, then the rest are better off doing the same.).  The only rewards for the politicians in this are media-time and vote bank support .


So what I do advocate is a mix of open data and predictive models plus persuasion and media (attention) frenzy that’ll overtake anything like the top-down media stuff the politicians can stir up. It won’t work without both of them, but I have no clue/idea about what will be successful and what will not in the latter case, so will focus majority of the post on the first.

Advocating open data access (water level, population, catchment area, drought area, cultivation area, predicted loss of agricultural area etc….) managed /maintained by a panel of experts, but open to all for debating and opining..

Major points(on the open data front):

  1.    Make the data open and easily  available . Here the data will be catchment areas, agricultural need estimates, actual rainfall, water table levels, water distribution wastage/efficiency, sand mining and their effects on water flow, economic impacts of the water shortage(bankruptcies, loss of revenue, loss of investment etc..). (There are some platforms like this and this already in India)*
  2.    Create/use a open data science platforms let bloggers and volunteers, modify the models (for estimates) and make blogs/predictions based on the given data, but with different models and parameters. (Some tools can be found here and here)
  3. Try to present the models in a way they can be interacted with even by people without programming experience. (The notebook links i provided above need python knowledge to edit, but anything built with this won’t)
  4.  Add volunteers to cross-check some of the data, like sand-mining, rain fall level, etc..
  5. Publish/collaborate with reporters to inform/write stories around the issue, with the help of models.(something with atleast the level of science journalism.)


Some thoughts(on the media – based persuasion front):

  1.  Recruit enough people interested in the exercise of figuring out details about impact of the issue.
  2. Make sure you can reach the ones that are currently most likely to indulge in violence(I can only guess at details, but  better targeted marketing strategy is what we need).


O.k: Enough of the idealistic stuff.

  1. Will this work? Well the question is too broad. Will it work to bring out the truth.. Ah it can bring us closer to the truth than what  we have.  And more importantly can define/establish a method/platform for us to get closer to data-driven debates and/or arguments.
  2. Will it cut the violence/bandhs/property-damage etc? Well that  boils down to the media and marketing front activism or work done. Leaving that persuasion part to politicians with skewing incentives towards gaining votes(from the steady voting population) is the problem now.  So can we have alternative parties (say,  business owners,) trying to use persuasion tactics only to discourage violence? I don’t know, but it seems likely that violence and martyrdom is preferred mostly by politicians and dons, but not the rest.(say media, local business owners, sheep-zens etc…). So this move has lower expected probability of violence.
  3. Who will pay for all this effort? Ah.. a very pertinent question. The answer is well, it’s going to be hard to pay the costs of even maintaining a information system, not to mention the cost of collecting the data.. That said, I think the big challenge is in the cost of collecting the data, and finding volunteers(something like this in US) to collect it for free. As for the hosting, building and maintaining an information system, I think there can be a cheap way found.
  4.  Is this likely to happen? Haha… no.. not in the next half century or so..
  5. Is there a cheaper way? Ah.. Not in the global/community/country level.. But at the individuals(media/politicians/public(aka u and me) ) sense yes, but it’s not really a cheaper way in the cost it inflicts. May be I’m just not creative enough, feel free to propose one, just be careful to include the costs to others around you now and others to come in the future.(aka your children)
  6. Why will this work? Well apart from the mythical way of saying “Sunlight is the best disinfectant”, I think this approach is basically an ambiguity- reduction approach, which translates to breaking down of status illegibility. (One reason no politician is likely to support this idea.) Status illegibility is the foundation of socio-political machinations and it applies to modern day state politics. So this will raise the probability of something close to a non-violent solution.
  • — I haven’t checked whether these data-sets  are already openly available, but I doubt they are and even if they are, some of the data are estimates, and we would need the models that made the estimates too to be public.


UPDATE: A few weeks after this I looked up on the google maps, the path followed by cauvery from origin to it’s end at the sea, and realized, I’ve actually visited more of the places it flows through in Karnataka and a lot fewer in TamilNadu. But that doesn’t change my stance/bias on misuse/abuse of sand mining and lake resources as housing projects in TN as that’s a broader , pervasive and pertinent issue.


UPDATE-1: A few months after writing this there was a public announcement, which if you read close enough is a typical persuation-negotiation move, with a specific action(and strong concession, and right now) demanded from the opponent, in exchange for a vague, under-specified promise in the future. This whole thing was on the news, is more support for my thesis that the incentives for politicians are skewed too much towards PR.


UPDATE-2:  Some platforms for hosting data, models and code do exist as below(although with different focus):

  1. Kaggle
  2. Drivendata
  3. Crowdai

so the question of collecting, cleaning, verifying and updating data is left.Also here’s a quora answer on the challenges of bootstrapping a data science team, which will be needed for this.