Truth

I was nearly finished with a 20 page paper (of sorts) on searching for bots in computer networks when I took a break and scanned the contents of my RSS feeds. This struck me as particularly timely and funny:

garbage_math_2x

As I told my boss last week I was disappointed in the algorithms used in what is considered “state of the art” tools. I actually found a strong inverse correlation in the “scoring” of network traffic of highly suspicious traffic compared to clearly normal traffic. The higher scoring traffic should indicate high probability of the traffic being communication with a Command and Control Server (C2 Server) and lower scores with normal traffic. I easily found instances where just the opposite was true.

When I used synthesized data I could get the expected scoring results but real world data demands new detection algorithms. It looks to me like bot builders also do research. Existing algorithms appear to be essentially garbage.

Quote of the day—RyanSepe

All this awareness would make us liable. Without them its ignorance, if we hire them it becomes negligence and I prefer ignorance.

RyanSepe
February 28, 2020
Suggested caption to this cartoon:


[There is way too much truth in this.

Companies have finite resources. They have to prioritize their cyber security efforts. If something is documented as an active issue, or even a potential weakness, and they don’t address it in a timely manner they have legal liability issues to deal with as well as fixing the problem.

In the “big picture” view of things companies have a lot of motivation to “not put it in writing” until they have the resources to deal with it. On the other hand, if managers don’t show they have a backlog and are overworked they aren’t going to get the resources to fix things in a timely manner. I have more than a little sympathy for cyber security managers caught in this dilemma.

After illegal computer access incidents have been made public Barb sometimes tells me, “I wish they would just stop doing that!” I would be out of a job, but the world would be a better place. So much money is spent on security that from a big picture you see it as huge waste of human and even natural resources (millions of computers monitor and guard against intrusion as their sole purpose). Even when the criminals are caught (extremely rare) they will never have to pay for all the resources spent in finding them and bringing them to justice.

And, of course, it’s never going to happen. Some of these criminals do it for the “free” money. Others do it for the thrill. And some do because they are spies in search of information useful to their country. There are always going to be those type of people. The best we can do is find them, stop them, and prosecute them if we can build a case against them.—Joe]

Mandatory “social distancing”?

Hmmm…

Gov. Inslee says ‘mandatory measures’ under consideration to combat coronavirus in Washington

Since the novel coronavirus emerged as a threat in Washington, officials have sought to keep people here from infecting each other by offering advice, health care and other assistance. What they haven’t yet done to slow the spread of the virus is tell residents what they can and can’t do.

That could change at some point, however.

Officials are considering mandatory measures for social distancing as part of the state’s effort to combat the outbreak, Gov. Jay Inslee said Sunday.

Barb, my oldest daughter, her spouse, and I, all in Bellevue, have been doing our part for the last week. We have been working from home and minimizing contact outside our homes. We are also prepared for several more weeks as needed.

We live in interesting times.

Pushing the limits

I like pushing the limits in certain directions.

Recently I have been spending nearly every waking hour working on my Bird Dog software for work.* I’m dealing with information on billions of network connections. I extract the stuff of interest and present it in an way which makes it easier to find the wood slivers in the hay stack. After using all the algorithmic tricks available I started finding places to do more parallel processing.

It was with great satisfaction that I found that I pretty much continuously keep all eight logical processers at 100 percent when doing certain tasks:

Limits

Each one of those processors is over 1000 times more powerful than the single processer I had on my first personal computer. And just the Bird Dog executable would take up over 75% of the hard disk space on that computer. Never mind the O/S or the database software which wouldn’t fit on a dozen hard disks I was so proud of at the time I first purchased it. “I’ll never run out of room on this disk!”, I foolishly told myself.

I now routinely open up text files in Vim for review and/or editing that are 50 to 100 times larger than what that hard disk could contain.

I like living in the future.


* I received an email from the company patent team earlier this week. They told me they are pursuing a patent on Bird Dog. I think the existing invention disclosure is okay, but the next one will be AWESOME! I’m really excited about what is coming up next. It’s as if a decade or more of my life’s work is coming to a focus on this one thing. I’ll probably need a more powerful computer, or set of computers, though.

Working from home

Health officials in King County (Seattle area) are recommending, among other things:

Workplaces should enact measures that allow people who can work from home to do so.

About 5:00 PM on Wednesday a blog reader told me::

Microsoft just told all employees who can WFH to do so until March 25

My employer said something similar yesterday. My team started WFH the day before that.

I can work from home for almost everything except meetings where someone is likely to be using a real whiteboard (we have virtual whiteboards in some conference rooms).

My first thought was, “Will the VPNs fall over?” So far both my MS contact and I have had not had any problems with our Internet connections to work. I suspect they have self-scaling VPNs.

Barb has been working from home exclusively for years now. It’s a little odd for both of us to be working from home every day. It’s nice but it just feels a little odd to only see each other for such extended periods. I wonder how it will feel after three weeks.

Yesterday I asked Barb if we are going to get “cabin fever” and get irritable or something. She thinks she will be okay as long as she doesn’t feel physically trapped as in being snowed in or something.

We’ll probably will go for walks occasionally. That should help and it should be safe as long as we don’t have contact with other people.

That’s odd

I’ve frequently read that important discoveries and inventions more often start with “That’s odd.” than with “Eureka!”

And so it was with a discovery of mine a week ago today.

I write software tools for the cyber security team at a major corporation. The culture is somewhat freewheeling. In the first couple of days when I started work my boss told me something to the effect of “People create their own positions here.” After the first couple of months I would talk to him no more than once a month. Sometimes it would be far longer than that. I did, pretty much, whatever I wanted. At review time I would be told, “We really like what you are doing and keep it up.” My model was look at what people around me were doing and write tools to make their job easier, faster, and enable better results.

I sometimes would joke that my goal was to eliminate the jobs of the people around me by writing the software to replace them. In reality what I did just meant people could be far more productive. Cyber security is never ending and I don’t see an end in sight for a job in this field as long as we have computer networks and human nature is what it is.

Nearly everything I did was little web application which would do things like check IP addresses for being on black lists and geolocate them, pull data and reformat from sensors, and graph data on “dashboards” for management to look at. My background task was working on something much bigger. I would see patterns in some of the data I was pulling from sensors and would try to get someone to investigate what I thought was suspicious activity.

The investigators would look at it for a few seconds and tell me something to the effect of, “I can see anything here. I need to see A, B, and C as well in order to know if this is anything.” So, a week or two later, I would show them similar data with A, B, and C added to the set. Again they would look at it for a few seconds, not see what I was seeing, and tell me they needed X, Y, and Z as well.

This went on for some time. I was somewhat frustrated and annoyed but I was learning how they did their jobs and what data they needed from multiple sources to evaluate a potential threat. But tens of thousands of rows in a spreadsheet with dozens of columns still didn’t allow people to quickly see the patterns I believed I was seeing. About two years ago I had kind of an eureka moment and I came up with a much better way of viewing the data (patent idea submitted to our attorneys was made late last year).

I started writing the software and explained it to anyone who expressed the slightest bit of interest in what I was doing. I gave the software the name “Bird Dog”. It essence it’s hunting through the grass and brush searching for specific things of interest to the (cyber security threat) hunter. It then points them out and then, when given the command, flushes them into the “air” such that only the the blind could not see them.

Everyone that sees it thinks it’s awesome but as much as I try I’m the only one that uses it. Everyone likes the data it produces but they don’t use it themselves. I think I need to make it easier to use but that’s a different story.

Last weekend I was putting in extra hours working on Bird Dog because I had gone through a major rewrite and it was to the point where things were working again as features were reenabled and new features were showing up. It’s very exciting to see what things will show up in the data with the proper visualization.

One of the things I had occasionally done in the past was to run a set of our externally facing IP addresses against the lists of “high risk” IPs. I didn’t have a complete set of our IP addresses but I had gathered some from public sources and had somewhat automated the process. I still had to copy and paste the list into a web app, click a button, and download the .CSV file into Excel. It didn’t take long but I never found anything and didn’t do it very often.

After the rewrite Bird Dog had a new data source. The new data source included more of our externally facing IP addresses. Bird Dog would now have not just my hand crafted list of IPs but IPs from the firewalls and other sources that might not be on any easily available list. And Bird Dog automatically added the risk scores to every public IP it saw, not just the IP addresses which were not ours (a previous limitation).

Last week during my testing of the new Bird Dog code one of our IPs was given a risk score indicating it was considered “Malicious”. That’s odd. I have been doing those sort of checks for years and I had never seen that before. But, it was one of the new features of Bird Dog and I knew it was possible.

I pointed it out to my boss. He and I spent a few minutes on it. We tried to find out why it was considered high risk but the supplier of the risk score for that IP had a 404 error on the web page for that one IP.

Another investigator was assigned and we looked some more. We didn’t make much progress and could create a story matching all the data that it was a false positive and we didn’t need to worry about it. We were about to close the case and move on when the vendor who had supplied the risk scores showed up for a meeting.

One of the guys (who plans to attend Boomershoot this year, BTW) stopped by my desk and asked how things were going. We chatted about Boomershoot some and then I told him I was a little frustrated about the missing risk score “evidence” for the one IP address. They get their information from various sources and had provided a link to the original source which where I was getting the 404 error from. He pointed out his company had cached the web page and we could just click on a different link. It wasn’t obvious to either I or the other investigator and we both missed it.

Together the vendor and I looked at the cached web page. We quickly determined that as far as our network security was concerned it was certainly a false positive. But the data was something we couldn’t ignore.

When my boss, a former police officer, came back to the office I showed it to him and asked if it should be forwarded to the police for investigation. He asked me to write it up and forward it to him and he would forward it to Corporate Investigations who handles all interaction with law enforcement.

Within a couple hours the referral had been made. Later that day my boss wrote an email to our director (some details redacted, indicated by XXX, for various reasons):

Joe was working on his Bird Dog code and identified a XXX IP address labeled “High Risk” by XXX.  After additional analysis, Joe and Mike found the IP address was listed in a cached webpage where someone posted XXX links to suspected child pornography

This was immediately handed off to XXX and the appropriate LE referral was made to the National Center for Missing and Exploited Children.

Great work by the entire team to keep digging and hopefully contribute to protecting a vulnerable child!

Neither I nor anyone I know clicked on the links. We all know better than to do that. You never want to go there.

This is probably the best, for certain measures of best, find so far by Bird Dog. And it was totally inadvertent. Sometimes it’s the odd things that are the most important to follow up on.

Quote of the day—Brad Smith

The pressure to put data centers in more countries is giving rise to what is rapidly becoming one of the world’s most important human rights issues. With everyone’s personal information stored in the cloud, an authoritarian regime bent on broad surveillances can unleash draconian demands to monitor not only what people are communicating, but even what they are reading and watching online. And armed with this knowledge, governments can prosecute, persecute, or even execute those individuals they consider threats.

This is a fundamental fact of life that everyone in works in the tech sector needs to remember every day.

Brad Smith
President and chief legal officer of Microsoft
September 2019
Page 45 in Tools and Weapons: The Promise and the Peril of the Digital Age


[One of Barb’s brother-in-laws recommended this book to me a few days ago as we were having a discussion about privacy and security.

I’m only about 20% of the way through the book but I’m really enjoying it. What I’m hearing matches the general tone of the culture when I worked at Microsoft. They take customer privacy seriously.

They have a team of about 50 people that work full time to respond to government requests and push back if the request is out of line with the law. They have promised to go to court rather than comply with requests that don’t have the warrants and documentation all in order. And they have gone to court numerous times. Smith claims they win in court 90% of the time.

I don’t know the details of the level of cooperation my current employer and the government have but I know that on the security side of things we take things very seriously. I also know that, IIRC, we have about 100 full time people who deal with government requests for information. I’ve talked with some of them and they too seem to believe it’s critical to keep the government on the straight and narrow.

I only see the criminal side of things but if we know or suspect customer personal information has been compromised, by either insider or outsiders, we put a stop to it as quickly as possible. And in the past year or two I’ve been seeing names of the people we chased end up in the news as being arrested, prosecuted, and convicted. None of them have been government officials, but that’s probably a little too much to expect.—Joe]

Project Diesel Memento

This morning I received an email from Ken in NH:

Raymond doesn’t name check you, but you are mentioned indirectly:
https://devblogs.microsoft.com/oldnewthing/20190930-00/?p=102942 

Also, I think you promised to tell us about the ammonium nitrate souvenirs some time ago. Any interest in writing that soon?

I sort of remember making that “promise”. I went to Raymond’s blog and found Ken had linked to my post on the same topic making it easier to find. Raymond makes reference to me in his post with:

One of the DirectX developers owned a farm and gave each team member a small plastic bag of ammonium nitrate as a souvenir.

You might think it odd he mentions the farm but not Boomershoot. This was in 1997, over a year before the first Boomershoot event. I was working on the explosives at the time but had not yet found something that worked. I had the ammonium nitrate but hadn’t yet figured out how to make it go boom with rifle fire.

Back to the 13 year old “promise”. It’s time to deliver.

I went looking for the 22 year old Word document. Yeah, I’m a packrat and knew I wouldn’t have deliberately deleted it.

I found two documents:

Directory of H:\Humor

09/30/2019  11:33 AM    <DIR>          .
09/30/2019  11:33 AM    <DIR>          ..
07/22/1997  03:56 PM            35,328 Diesel.doc
07/22/1997  04:00 PM            19,456 Diesel2.doc

Diesel.doc:

Project Diesel Memento

Enclosed you will find about 1/4 pound of ammonium nitrate (N2H4O3). When mixed with approximately 1 teaspoon of diesel this common farm fertilizer could be made to explode with the force of several sticks of dynamite. If the charge were properly placed in the ground it would probably make a hole about large enough to bury a person (or two in the case of Raymond Chen).

As it stands, and even if mixed with diesel, it is extremely difficult to detonate. On the farm we used 1/2 stick of dynamite which itself required a blasting cap to be detonated. It requires extremely high pressures (several thousand PSI) and heat to detonate. On the farm we would bore a hole in the hard dirt and clay under a stump and firmly pack in about a pound of it (mixed with diesel and the dynamite booster) to “liberate” the stump from the ground.

Other means of detonation exist, see for example New and Improved C-4 — Better-Than-Ever Recipes for Half the Money and Double the Fun by Ragnar Benson or The Anarchist Cookbook. I have extreme doubts about the accuracy of the recipes in The Anarchist Cookbook and have been unable to get Benson’s recipes to work when attempting to detonate the mixture with rifle fire (as he claims will work). So, if you want to use this to blow something up you will probably have to obtain some dynamite and blasting caps or ask Timothy McVeigh for help (I believe he actually used nitromethane, not diesel).

Regards,

Militia Joe from Idaho

The Militia Joe from Idaho reference is to a skit I created for a Swine Before Pearls television show in 1995 while I was working at Microsoft on Direct X 1, The Manhattan Project.

Diesel2.doc:

July 22, 1997

Raymond,

One is for you (if you want it). Please give one to NWilt. The others you can distribute as you see fit (trinkets for the natives, whatever).

Have fun!

See you soon. You are coming over for harvest, right?

Regards,

I printed something like 20 copies of “Project Diesel Memento”. I put about a quarter pound of ammonium nitrate in “Seal a Meal” (as they were called at the time) bags. This was sealed off in the bottom part of the bag and the printed text was put in the top part of the bag and sealed. I then shipped the entire set to Raymond at Microsoft with the cover letter shown in Diesel2.doc.

NWilt was my manager when I worked on Direct X 5, Project Diesel. It was probably about 2012 when NWilt contacted me and invited me out to lunch. He told me he still had the Project Diesel Memento.

And now you know the mementos story I “promised” to tell so long ago.

Mugme Street news

I used to work downtown Seattle in the Century Square building which has one side on 3rd Avenue between Pike and Pine streets. Barb calls this street “Mugme Street” because all your warning flags go up as soon as you poke your head out of the building along that one block length of 3rd. When I was working there I blogged about it fairly often.

Last night there was a “dispute” at 3rd and Pine which moved below the street to Westlake Tunnel (and bus station). The end result was three people shot and one dead:

Police are looking for a gunman who shot three people, killing one of the victims, near the Westlake light rail station in downtown Seattle Friday night.

Authorities said three people were struck by gunfire about 9:30 p.m. Two of the victims were found near Third Avenue and Pine Street. The third victim was at Fifth Avenue and Pine Street.

It is important to note that the locations were all underground at Westlake Station.

Police believe a dispute started above ground near the McDonald’s on Third Avenue and moved below to the Westlake Tunnel. They also believe there is only one suspect.

I used to take the bus into town and get off at Westlake Station.

That McDonald’s has to be one of the most dangerous to eat at in the country. I almost never got food there and it just really bothered me to even linger in the area.

I’m so glad neither Barb nor I work downtown now.

Update: A suspect has been arrested: 20-year-old Westlake Station shooting suspect held on $2M bail

Overheard at work

I work in computer security. I write software to search for “interesting” data in billions of connections between millions of computers. Many times the “interesting” stuff I find turns out to be not quite as “interesting” as I initially thought. I always run it by others to do a “reality check” before investing too much time investigating or raising an alarm of some sort.

I showed my boss some “interesting” data recently:

Chris (my boss): Do you every feel like that guy in a movie sitting in front of radar screen saying, “I don’t think that is a flock of birds!”?

Me: All the time.

Chris: Yeah, well, I don’t think this is a flock of birds.

Overheard at work

Caity: Do you remember the time we had to write the report on the electromagnetic pulse? A solar storm or something?

Joe: I remember the report on the possibility of North Korea setting off a nuke and creating an EMP.

Kelsey: I remember that! That was really depressing. It was like, “Time to go to Idaho and hide in a bunker.”

[Laughter from everyone and they all look at me.]

Joe: I don’t have a bunker in Idaho! I may have an explosives production facility in Idaho but I don’t have a bunker.

[laughter from everyone]

Joe: It’s all about offense, not defense.

Quote of the day—Jodie

I think everyone should get as much sex as they can.

Jodie
October 23, 2018
[Jodie is my boss.

She has also been known to announce, “This is now a HR free zone.” and say something less that politically correct.

I have an awesome boss in a number of ways (never mind that she once tried to drown me). I’ve never had a boss tell their staff this sort of thing before.—Joe]

New shooter report

We have a relatively new intern on my team at work, Nashwa. She grew up in Texas and speaks fondly of it so I figured she was at least comfortable around gun owners. I had taken everyone else on the team, except my boss Jodie, to the range but not Nashwa.

I have invited Jodie many times. While she expresses great interest she has not found a time slot that works. I give her a pass because she recently finished up training with the FBI where she learned to shoot everything from handguns to sniper rifles. I’ll get her to the range someday but today was Nashwa’s day.

I had the training bay reserved just for the two of us from 4:00 –> 6:00. It turns out she had never fired a gun before. I asked if she was right handed or left handed. “Right”. Which eye is dominant? “Right”. I was a little surprised she knew. My surprise must have shown because she then said she wasn’t sure. I did a quick test and found she was left eye dominant. I first taught her shooting left handed and then part way through switched to right handed for a while. She decided to stay with left handed shooting.

I started her out with dry firing of a Ruger 22/45 Light with a suppressor. She looked like she had it down. But her first half dozen real shots were all high. Nice group. But they were about three inches high at 10 feet. I went over sighting again. Still the same problem.

20180830_165000

I fired a few shots. It was maybe a quarter inch low at that range.

We went over the sighting again. “Oh, I wasn’t really looking at what was going on with the rear sight.” Hmm… I’ll have to work on how I explain sights.

I gave her a clean target and she was putting them just below the bulls-eye:

20180830_165250
Ahhh… Yes. The new shooter smile.

I moved her to shooting a simulated steel match with four targets on one piece of paper and removed the suppressor.

She was getting all five hits in under ten seconds.

Next I gave her Major Power Factor loads in my STI DVC Limited. With essentially the same results. But after a few strings the misses started increasing and getting more and more wild. It was time to go back to the .22.

20180830_171819

She still had some misses. Back to dry fire. We needed to end the day on a positive note.

The dry fire looked good. I pretended to put in a loaded magazine and she “fired” again. There was some serious movement of the gun when she pulled the trigger. More dry fire. And then, finally, live fire. She was back to consistent, solid, hits  I shouldn’t have let her fire so many rounds through the .40. She was starting to develop a flinch.

After we cleaned up and packed things up we talked a little bit. She had two questions:

  1. Q: How much do I owe you?
    A: Nothing. The first time is free for new shooters.
  2. Q: How often do you come here? I would like to go again.
    A: Two or three times a week. But you don’t need for me come with you. You can come here by yourself if you want or bring a friend anytime they are open.

We now have a new member in the gun community and a team member at work that fits right in.

Overheard at work

Some of my teammates and I were discussing the details of an email we got from someone who claimed they had been hacked. It had a number of conclusions which were absurd on their face and the data they supplied were consistent with an alternate hypothesis which was void of any wrongdoing. Yet, we were inclined to look into it a little bit more…

Joe: What they are saying doesn’t make any sense but it’s all within the realm of standard ignorance.

Caity: I like that phrase, “Within the realm of standard ignorance.” Can I be Queen of the Realm?

Another ASI match

Last Saturday Ry and I went to an ASI match at the Renton Fish and Game Club. This was my third and Ry’s first match of this type. We were not happy with a few safety issues that happened with our RSO officers. There was no one in real danger but some rules were broken and contrary to every other match I have been to they blew me off (in a friendly manner) when I gently pointed out one of them.

The match itself was good. The stages were interesting enough yet simple such that beginners wouldn’t have a problem with them. I came in 10th out of 65. If I hadn’t just barely nicked a no shoot target I would have came in 6th. And it annoys me they assigned the penalty as a procedural on a different stage (no difference in my final score). And they also misspelled my name. But that’s minor stuff.

I wouldn’t bothered with making a video but I had invited my team at work to watch and/or participate at the match and Caity told me that she and Kelsey were going to some sort of women’s conference. I joked that the match would be more fun. She joked back that she would take pictures and we could compare on Monday. So… I had to make a video:

Shooter POV Action Shooting International Match from Joe Huffman on Vimeo.

Caity took one boring picture. I won.

Overheard at work

Today, from a meeting at work (redacted and paraphrased as needed):

Jodie (my boss): Ms. “X” and some other adult entertainers have contacted Mr. “A” and have starting talking. We should reach out to Mr. “A” and get a relationship going so we can correct any false or misleading information he gets from other sources.

[Joe starts smirking]

[Jodie looks at me and stops talking]

Joe: It might be difficult to establish a competing relationship when our competition is a bunch of porn stars.

Fortunately, everyone in the room seemed to think it was as funny as I did and I wasn’t sent to HR for reeducation.

Quote of the day—Devin M.

They seem to be legitimate illegal activity.

Devin M.
May 3, 2018
[This was from work.

Devin was researching a business that, essentially, sold stolen goods and had a good reputation with their customers.

This is sort of like an “honest politician” is one which, once bought, stays bought.—Joe]

We were just talking about this at work

There are a three new people on my team at work. Two of them have a decade or more of experience in the field and one is in sort of an expanded intern program and is “drinking from a firehose” as she is coming up to speed. A week or two ago the newbie expressed some insecurities about her being able to contribute and one of the experienced guys reassured her and told her about “Imposter Syndrome” and said that he feels that and probably everyone does. Nearly everyone on the team jumped in to reassure the newbie that she is doing extremely well (she is) and had their own little stories about how they feel insecure about various aspects of their ability to do their job.

XKCD gives us another example:

Impostor Syndrome