Swarms

I haven't hidden my love of distributed generation.  There's something great about seeing a clear vision for the future and knowing that, while it may take some time, society's progress towards this vision is inevitable.  As one of my high school football coaches liked to say to us when we were enjoying the last few moments of cold air conditioning before going out into the sweltering Florida heat, "Get your asses off that bench, there's no point in delaying the inevitable."


That's one of the best things about being in the solar industry.  It doesn't matter how much money big oil/gas throws at ads that make solar advocates look like radicals, you simply cannot stop something that makes sense economically and environmentally.  People are starting to come around and it will only intensify in the future.


As distributed generation evolves, it's only logical that the future of the utility industry is in localized power management.  It happened with computing, it happened with cell phones, it's going to happen with your electricity.  Getting your power from one, big, central source that is very far away will become as outdated as landline telephones.  Houses are going to be generating their own electricity, storing it to use later and sharing it with their close neighbors when there is excess (and get paid of course).  You won't need to worry about the power going off for the entire city because a squirrel chomped on a power line 200 miles away anymore.


In one of my favorite TED talks, Steven Strogatz talks about how we and other animals naturally sync (not the Bluetooth kind of way).  In it, he has mesmerizing footage of swarms of birds beautifully moving through the sky together.  They look like they're being orchestrated by a single maestro or even central nervous system.  But they're not; there's no one leader or director, each bird moves independently but in unison as a group.  He says swarms only have 3 simple rules:

1.  They can sense what's around them

2.  They like to form lines (be ordered)

3.  They like to be close to each other but still have some space in between


When predators try to attack a swarm, those in the group don't know exactly what is happening, they just know that they neighbor moved out of the way quickly so they in turn move out of the way.  Their neighbor made a sharp left, so they made a sharp left.  The predator attacking the middle is left with a mouth full of air because the swarm automatically avoided it, without even knowing what was going on.  A single leader's job would be incredibly complicated, instead the swarm knew exactly what to do without even communicating to each other because they just followed those 3 rules.


I bring this up because localized power management will eventually do the same thing, it will act as one swarm but with no single leader.  Except in this case the predator is power outages, it could even be clouds.  Yes clouds.  Shade is the enemy of solar power and clouds passing overhead cause intermittent power fluctuations.  Just like anything else, stability and consistency is the name of the game so these variable changes cause problems in power management.  


But don't worry, we're just going to take notes from Mother Nature on how to combat this.  We won't need a central player or advanced algorithm to predict what will happen and direct the power to different houses depending on its usage.  The houses will all communicate with each other, when one needs more power it will be sent some, when another is temporarily shaded it too will be helped by other houses that aren't shaded.  Localized power management will act as a swarm, each unit acting independently but the group as a whole will be one efficient system.  And it will follow the same 3 rules that birds and fish have been using for millions of years.


"I would feel more optimistic about a bright future for man if he spent less time proving that he can outwit Nature and more time tasting her sweetness and respecting her seniority."
- E.B. White


// Edited because the first draft had a couple mistakes

What a Good Remix Does

For some people, when they see the word "remix" next to a song's name, they automatically dismiss it.  For me, it's the opposite.  I think remixes, when done properly, can add a great deal to an already great song.  Sometimes it just enhances the original, sometimes it completely creates a different song.  Here are two examples of remixes completely changing already great songs and, in my opinion, creating better versions.




And...



Using Census Data Analysis to Find Targeted Markets

That's a SEO-approved headline if I've ever seen one.


Thought it'd be good to share a research project I did about 8 months ago for my Data Analysis class.  Maybe someone else can duplicate it in another industry and benefit from it.  The basic instructions for the project were to use exploratory data analysis to gain a business insight; it was an introductory class for business professionals so it didn't involve very heavy lifting but it was challenging nonetheless.  Well since I concurrently was working (and still am) for SolarCity, I thought it'd be good to incorporate it because there might be benefits that emerge throughout the process.  Of course, I won't disclose any business information.


My goal was to pick a regional warehouse and be able to find the most "attractive" areas in which we should focus our sales efforts on.  First I picked the smallest area that the US Census gave open data for (which was the zip code) and found the top 50 zip codes that we have current customers in.  My theory was that our future customers would more likely be similar to our current customers than not, so I decided to focus my data analysis on describing our current customer base.  The US Census will give you a shitload of data, as long as you painstakingly gather it in little chunks.  So that's what I spent most of my time on.


Below is an example of the finished product of collecting the data individually and organizing all of it.  I'd say this took about 2/3 of the total project time:


First I just played around with a portion of the data, to make sure it made sense before proceeding.  To do so, I did a X-Y scatterplot of each Census data column against the total count of customers in each category; thus a positive correlation would mean you would expect more customers in a zip code the higher it is in that certain Census variable.  I only used the relative percent of some of the Census data columns, since this would paint a better picture than total numbers for certain ones.  Below is an example of linear regressions of some, with neutral, positive and negative correlations:


This yielded some interesting patterns.  I wanted to further analyze it and divided each variable (column of a particular statistic from the Census) into similar groupings.  Yes I added some personal bias to this analysis, but rarely was it hard for me to pick a particular grouping for each variable, so I'm confident I didn't inject too much bias.  Below are the groupings of the categories of different Census data:


Then I found the correlations by grouping, this was particularly interesting to see what had the highest positive and negative indicators of more customers.  Below is the graph by grouping:



Within each grouping, I was able to create bar charts that showed the correlations as you went across the spectrum of each.  So if you graphed house value, and there was a clear differentiation line where it flipped from positive to negative correlations, you'd know up to what house value to advertise to because anything more and you're wasting your efforts.  Below is an example of one of those charts, and it's not house value:


The final step was then ranking the zip codes I studied by the number of predicted customers it should have.  This would show the most "attractive" zip codes in which to focus on.  Also, my finding the difference between actual customer count and predicted customer count, you could find zip codes that are "underperforming" and target those first, since they should automatically have more customers than they do.  Below is an example, the right most column conditionally formatted red would be underperforming:



I thought the final product turned out great; I got a good grade on the project but more importantly I hoped it would help increase SolarCity's sales efforts by targeting specific areas that are more "attractive" than others.  It would be a quantitative compass, not a perfect solution but better than before.  After vetting the project with my professor and trusted confidants to make sure it made sense, I approached our sales managers and Direct Sales team (they're the ones that go around neighborhoods on foot so I thought it'd benefit them the most to have data to back up their choices).  However, after initial enthusiasm, it fell off the map like other ideas.  It would've required a bigger and more accurate test and pilot project to prove its worth before rolling out, but I didn't see how it could hurt our efforts.  In my mind it was worth a try.  But as it is often, there are different things on the immediate agenda, either that or I didn't talk to the right person.


Hopefully it's revisited in the future and I can expand on it more, this time while getting paid to do the analysis.  That'd be cool.  But I encourage anyone else who thinks they can benefit from similar analysis to do the same and see if it helps you out.

Edison's Inspiration and Revenge


I just started reading "Empires of Light:  Edison, Tesla, Westinghouse, and the Race to Electrify the World" by Jill Jonnes and I'm already enthralled in it.  Although she can be superfluous at times (and so was using that word), she paints an exciting story about the battle a century and a half ago to bring electricity to the masses.


We all know the ending (Tesla/Westinghouse and AC power won out, even though it was more dangerous because it could travel long distances more efficiently) but that's what makes it so interesting.  Solar is going through a similar battle right now, and it is on the Edison and DC side (solar panels produce DC energy, and right now that has to be converted back to AC power to power things in your home -- hysterically it is actually converted back to DC power to power a lot of electronics) but updated to today's technology.  He was an advocate of a distributed system of power generators; unfortunately due to the technology of the time there had to be a toxic, noisy generation station every couple city blocks.  But in today's time, that is just the same exact thing as quiet, clean energy producing solar panels on your home's roof.  They don't put off any toxic fumes, they don't make any sound, all they do is just sit on top of a small area on your roof that was previously just occupied by dirt buildup and bird shit.


Anyways, I'm sure this won't be the only post I write about solar and the "War of Currents", but it hit home when I just read about Edison's inspiration for his invention of the lightbulb.  Jonnes writes, after Edison had just been introduced to arc lights:

"Then Edison rushed back to quiet, bucolic Menlo Park, his research workshop in backwater New Jersey, to throw himself into creating a better and more practical electric light.  He worked feverishly, thrilled at the possibilities of this new field."

But the best quote comes from Edison himself, describing where he got his inspiration from, as Jonnes continues:

"It was all before me.  I saw the thing had not gone so far but that I had a chance.  I saw that what had been done had never been made practically useful.  The intense light had not been subdivided so that it could be brought into private houses."

Jonnes then wrapped it up best with:

"Edison always liked to go after 'big things.'"


I felt connected to those words by Edison, because that's exactly how I've felt working at SolarCity.  Solar is not a new technology; but widely-distributed, affordable solar is a new thing and we're working to help bring that to "private houses".  It's all before us, whether SolarCity makes it long term, or the industry evolves and the next big player comes to dominate.  The world will benefit from this "Solar Race" the same way it did with the "War of Currents" bringing electricity to the masses.


Jonnes goes on to describe how Edison got it right:

The man who came up with the best arc light system might well make a fortune stealing away even that 10 perfect of the gas lighting business -- that of streetlights.  But the man who could subdivide the light -- to take it indoors and tame it into a gentle glow -- and power it with a dynamo, he would be the true Promethean, the blazing electrical pioneer, the hailed benefactor of humankind (and wealthy to boot).


Solar is doing that; we're "subdividing the light" by producing clean power right at the site that it is being consumed, or close to it if you're producing excess energy when you're not home during the day.  Right now, we're just beginning to come close to the price of power generated by fossil fuels.  But coal, oil, natural gas -- they can't come close to where we will be producing power, right at each home.


So that was Edison's inspiration, you might be wondering what his revenge could be?  Well, it's likely going to turn out that he was right all along with distributed generation, he just wasn't alive 150 years later to see it overtake AC power grids today and in the near future.  But with a much better technology than is coal-burning power stations of his day and age, distributed generation is inevitable.


Well, Edison will be right at least until fusion comes along.

Introduction to ideas

Obscure title yes but it'll do for right now.  


Often we go through life wanting to do a lot of things:  we want to be successful in school and our careers, we want to have a loving family, we want to maintain our health and exercise, we want to give back to our communities/religion, etc.  But rarely do we have time for all of them on a consistent basis.  All along, we also have a lot of ideas of different ways we think the world can improve but again lack time and the resources to fulfill the execution needed to bring said ideas to life.


Sometimes we tell these ideas to others to seek external support; sometimes we keep them a secret because we're scared of external judgment.  But for every idea we don't let be known, there exists the possibility that the world never got to benefit from a possible "missed connection" of mutually beneficial or associated other ideas.  


I've never been shy about sharing my thoughts if I think they can help improve things in any way.  Often this information has already been brought up and discussed or dismissed but that doesn't mean it shouldn't be shared.  You either show that you are on the same page as those who you are presenting the topics to or you show that you need to learn more about the topic.  But that shouldn't deter you from potentially sparking another person's interest to continue where you left off.  It's a pretty high risk, low reward process as long as you have vetted the idea internally as well as externally with trusted friends.


So that's why my first idea after introducing my introduction to them is for someone to create said idea-curation-then-creation website; someplace that builds off of the positive purpose behind message boards and serves as a melting pot of associated ideas with simple UI and transparent philosophy.  After allowing for ideas to be connected, the best solutions will bubble to the top through Reddit-esque upvotes/downvotes then a representative can be chosen and the project can be paid for through crowdfunding.


That's got to be a net positive for the world.

I <3 Hypem


I've been using this music streaming service called Hype Machine for a little over 5 years now and my account and all the songs I've hearted ("liked") over the years is easily one of my top 5 online possessions.  It's become a daily part of my life; the process discovering new music throughout the week is enhanced by always having one place where everything can be found.  This is going to make me sound dated to any other hypem users (cue the old grandpa voice) but I remember when top songs on their popular page only had a couple hundred likes, now their #1 songs (such as the Odesza remix of Zhu's "Faded" mentioned below) routinely have 5,000+ likes.


It's free to join and they're not very obtrusive.  You "pay" by having ads pop up all the time when you open your page as well as some tricky border ads; annoying but again, it's totally free, so it's worth it to me.  I'm sure they also get some revenue from referring people to buy concert tickets on songkick.com (which I've done many times over the years).


Soon they'll have an official Android app and it'll only increase my use of the site I'm sure.  They are pretty aware of their product niche but are always rolling out new features with an eye towards the future.  One reason why I originally liked them is because their popular page had a shelf life of 3 days so it always kept recent hot songs coming through.  Recently they rolled out the ability to create "lists" of different styles/energies of music, they are "Up", "Down" and "Weird".  I'm still figuring out the feel of each but this is brilliant on both the customer's and the company's ends; I get to have a quick way to play a similarly styled playlist and Hype Machine gets users to crowdsource genre-definition of all their songs.  Once again the wisdom of the crowds will prove to be very useful.


Check it out particularly if you enjoy electronic/techno/EDM/whatever you call it and indie, as those are primarily the genres I see represented.  I suggest when you begin to regularly use it that you curate a followed list of some blogs that you see playing the specific type of music you enjoy at the moment.  If the blog's new song isn't exactly your flavor, a lot of times there will be another song in the post that you will like.  

Suga Suga

I've heard a lot of good music recently but this one really stands out.  Partly due to timing as it's the beginning of summer, it's mostly good because it retains just enough of the original to bring back memories of summers long ago.  That and everything it adds is perfectly placed.  Would make a great song choice if you happen to be cruising down A1A in the bright Miami sun, just saying.


Baby Bash - Suga Suga (Royal Refix)


Dan Ariely Speaks the Truth

I was introduced to Dan Ariely when I picked up his book "Predictably Irrational" a couple years back.  I know it wasn't anytime recently because: A.) It's a physical book, not a wireless download & B.) The sticker on the back says it's from Borders.  Anyways, it was one of my first behavioral psychology reads and really sparked by fascination with the subject.  


It's always an exciting discovery when you find something that you quickly become obsessed with.  Learning how and more importantly why we (or at least Westerners that participate in most of the studies) think and act quickly became addictive.  I still find myself buying some of the intriguing and popular new releases even though most of them just recycle the same knowledge, and cite the same studies, but applied to a different scenario.  This isn't rational but it's still something I continue to do.


Back to Dan, he has a great blog that he frequently updates with real world examples of these behavioral psychology principles by taking reader's questions and trying to explain their thinking.  A recent post of his included this wonderful response:


Dear Dan,

Early in my career, I wrote a massive Excel macro for the large bank where I worked. The macro (a set of automated commands) would take a data dump and turn it into a beautiful report. It took about two minutes to run, with an hourglass showing that it was working away. The output was very useful, but everyone complained that it was too slow.

One way to speed up a macro is to make it run in the background, invisibly, with just the hourglass left on-screen. I had done this from the start, but just for fun, I flipped the setting so that people using the macro could see it do its thing. It was like watching a video on fast forward: The macro sliced the data, changed colors, made headers and so on. The only problem: It took about three times as long to finish.

Once I made this change, however, everyone was dazzled by how fast and wonderful the algorithm was. Do you have a rational explanation for this reversal?

—Mike

I’m not sure I have a rational explanation, but I have a logical one. What you describe so nicely is a combination of two forces. First, when we are just waiting aimlessly, we feel that time is being wasted, and we feel worse about its passage. Second, when we feel that someone is working for us, particularly if they are working hard, we feel much better about waiting (and about paying them for their effort). Interestingly, this joy at having someone work hard for us holds true not just of people but of computer algorithms, too.

The life lesson should be clear: Work extra hard at describing how hard you work to those around you.


I love it, so logical and true.  The bold part at the end is my emphasis on the underlying principle; it's partly sad that this is the main point of his answer but it's 100% true.  Many people and services should take his advice if they wish to increase their customer satisfaction levels.

"Coaching $$$ in Black & White"

I was scrolling through the Miami Herald's Sports page and a blog on FIU football by David J. Neal caught my eye.  Initially I noticed it because the second half of the title of the post said "Coaching $$$ in Black & White".  I didn't know if Neal was using "Black & White" metaphorically or literally; whether he was talking about the blatant nepotism on FIU's staff or if there was a pay disparity between black and white head coaches.  I assumed he meant literally because he capitalized Black and White, likely referring to the skin colors.  So I decided to look up their bios and compare them.


Here is the table of coaches I compared, it's most of the main assistants for the 2013 and 2014 seasons:


I couldn't find public salary info on their special teams coach from 2013 on the Florida Has a Right to Know Mr. Neal mentioned in his post so he was omitted.  I also left the head coach Ron Turner off the list because his salary would've skewed the averages.  The salaries were average for what info I could find over the last 2 seasons.  The disclaimer with this quick analysis is that the sample size is pretty small so we can't know any conclusions for sure.  But from the numbers we do have sure show an inflated salary if you are either white and/or related to the head coach or one of the coordinators, particularly if you fulfill both categories.


Here is the table of salary info by skin color.  The higher numbers in each column are considered "better" so they're highlighted in green (BTW how many promotions has Excel's "Conditional Formatting" gotten people over the years?  Has to be in the thousands):


It's pretty clear that while the black assistant coaches have 25% more years coaching experience (15.3 vs. 12.2), they are paid 13% less ($101,250 vs. $116,916) in salary.  Also white assistant coaches get more than DOUBLE ($20,046 vs. $9,076) the salary per year of coaching experience and make 1.7 times ($29,537 vs. $17,444) in salary per year of college coaching experience.  Those facts are pretty hard to ignore.  So what I assumed Mr. Neal was referring to, that white assistant coaches at FIU under Ron Turner make more than black assistant coaches, does indeed look to be true.


Here are some really fancy pictures in case you prefer that medium:


Ron Turner's brief but rocky tenure as FIU head football coach has also been controversial in that some people accuse them of favoring hiring direct relatives for assistant jobs and paying them more than people that aren't related to the top coaches.  Both Ron Turner and his offensive coordinator Steve Shankweiler have sons that are prominent assistant coaches.  And both of these sons share something in common when you break down their salaries as well.


Here is a table showing the same numbers as before, but broken down by if when you were a direct relative (in this case a son) hired as an assistant coach at FIU:


Again, it's pretty clear that there is some favoritism going on here.  If you were hired on as an assistant coach and you were NOT a direct relative, even though you had 3 times the coaching experience (12 vs. 4), you were expected to make only 8% more than if you were a relative ($107,722 vs. $99,500).  Even worse, if you were a direct relative, you make 2.7 times the salary per year of coaching experience ($34,166 vs. $12,595)  and make 3.7 times the salary per year of college coaching experience ($62,100 vs. $16,779).  Insane!  How is this allowed to happen?


Again, here are those numbers in graphical form:


Something is fishy at FIU, that's for sure.


//Edited to fix wording