Algorithms

Truth is not a function of popularity.

Fact checking is a matter of looking closely – scrutinizing. Scrutinizing is something most people have little time for. We are in a rush. We will have our truth while others have their truth. We must move along.  We like to have our conclusions ready-made. It is convenient to have a version of reality pre-packaged for us. We fly away like birds so we can look at the world down below with a singular worldview, find our prey, build our nests and mark our territory. Tweet tweet.

Birdwatch

I’m not sure what Twitter’s Birdwatch algorithm is all about. I’ll wait and see before saying much, as it is still in a beta-testing phase and will probably change for the better in the coming months. Apparently, a certain popularity level in its algorithms has something to do with whether the Birdwatch fact check gets promoted to the status of any real fact check. The idea is to crowd source a fact check.

It’s a cool concept in a way but it’s obviously got a long way to go. The backlash from a recent fact check offered by Tim Pool, a conservative, highlights the present state of the circus. If the tweet isn’t removed, you can search the link and scrutinize it for yourself straight from the source by clicking here. Whether you agree with Pool or not, it’s a fascinating subject for anyone interested in helping develop something like the CounterChecker app.

As for me, I will stick to highlighting social media created fact checking widgets from time to time as a point of contrast. I do this in order to help solidify for interested parties what my own vision is. It goes way back to when I was building a better search engine in 1998, which we later called the Ghost Machine.

My goal at that time was to improve the way search was done. I am an idealist and while the Internet held out much promise, the web also had what I thought were some serious flaws. At the time, the leading search site was Yahoo. It was typical in those days to search by category. There wasn’t as much to search for so you could do that back then, but very quickly search by key word replaced topical searching.

My complaint was that porn sites, scammers and other savvy web manipulators were using search algorithms to promote products and services, including sites loaded with computer viruses, to top search results in ways that were unfair and often dangerous.  To solve the problem, I offered freebies to phantom web shoppers we called ghostsurfers at ghostsurfers.com. Our ghosts had halos for a reason. This was our motto. We were there to improve what the Internet was. We were among the first to ever crowd source Internet ratings. An improved rating on a web site, would lift the result up to the top of the search results, using our algorithms. And like most search engines, our plan was to monetize the site through advertising.

I also looked at the Internet as an employment opportunity for the many. I firmly believed in revenue sharing and crowdsourcing, even then. So, we worked out a plan for sharing 50% of our advertising revenues with our ghostsurfers. The more traffic they generated to our site, the more they could earn, and this was the key to its viral growth. Although the site came crumbling down after the tech bubble popped in the year 2000, the concept was sound. We were enjoying over 1 million page views per day in the months of June, July and August of 2000. This was equivalent to a 5% Internet reach at that time. Not bad for a guy with never more than two employees working from his bedroom. My hopes were high, but I was unsuccessful at convincing prospective investors that the project was worthy of putting more of their money into.

The Real Me

In some ways that seemed fair. I was more of a do-gooder than a profiteer. My philosophy was, and still is, that if you do the right thing, using the right formula, the money will follow. I had a good revenue plan, but conveying the value of a flawless system was not something easily accomplished. It required communicating the nuances of the checks and balances that went into the design – to make sure no cheating and manipulation could take place. Things that were important to me as a do-gooder and a designer, were not the things that angel investors had the time to sit down with me and discuss – not while they were losing money on everything in tech.

I was easily dissuaded from pursuing my vision at that time. I had no prior tech experience beyond what I had taught myself. I had been a music composition major in college. I was working at a bank. My greatest strength was in theology. I completed a seminary degree before Dad died. My father was an entrepreneur living in Palm Beach, and knew all the people with money. He was always a help. Mainly, he saw me as a spiritual person. He knew the pain of business failure and how mind-consuming running a business can be. He’d had several heart attacks due to the stress. He stopped smoking way too late. Most of our conversations involved him talking me out of going into business. My older brothers did the same.

As I saw it, they were right. I had never made any significant profit on my previous business concepts. For instance, one of my start ups was Gofers. I could see big money in the future with a network of shoppers doing home deliveries. It was a great idea. But great ideas are meaningless. What matters is executability and talent. Did I put what little money I had into buying some cheap cars supplied with CB radios? Did I give them to friends all ready to dispatch? All I needed was a bit of advertising money to let the public know they didn’t need to go to the store any more. Just call me.

Really? I almost did it. I think that was 1981. I bought my fiancé a car with a CB radio. I coordinated shopping with her for our friends, but that’s as far as it went. When I found out the cost of advertising, I did a risk assessment, and backed out. It takes a lot of money, and great insurance policies, to start businesses. I was just happy to take a tax deduction for the jalopy I bought a pretty girl that I was in love with.

Oh the stories I could tell, but they weren’t about my amazing successes in business. I was under-confident. My family was telling me that I would never go anywhere and never do anything. And I believed them. How sad. It hadn’t occurred to any of them that theology and vision are a perfect match, and precisely what is needed in a worldly world. Worldliness was Dad’s specialty. He didn’t just know everyone who had money in Palm Beach. He had mob ties I didn’t know about. And he wanted to keep me away from it. He wanted me to know nothing about it, learn nothing of it and take no part in it. I didn’t.

Accountability

The key to good algorithms is checks and balances for which there are no work-arounds. The essence of good theology, is truth. When Jesus was asked who he was, he said he was the truth. “What is truth?” Pilot asked.  That’s a great question. I don’t really know what the truth is but I sure don’t want to hide it. If it damages someone, that may be something to think about. There is a place for discretion. Truth without kindness can be cruelty. But hiding the truth is for political teams to work on. What is in the public’s interest is another matter. Knowing the untwisted truth, even if through an examination of a binocular spin, is the place where each person has a right to decide. Which eye is correct? The left or the right? Does the truth lay in the middle somewhere, as Aristotle and Aquinas thought? The left and right eye correct one another. Sweet. The world sees more clearly that way. It would seem to be in our best interest to have more accurate perception and vision.

Correcting processes go much deeper than left versus right though. In the Ghost Machine’s search engine, a web link would be promoted higher in search results if one of our ghostsurfers ranked it that way, but only if they had high reputation points. Members were not rewarded for promoting links. They were rewarded for the quality of their ranking suggestions for whole categories. If they thought link A should take a higher rank than link B in a category, they didn’t just click an UP button for that link. They argued for why. They did so in a forum where a conversation about it could take place. And when they argued for why, they compared that link with the links its promotion would demote with other members. The group goal would be to arrive at the best ranking for that search term or category.

And … that was not enough. Stop and think. What might the vulnerabilities be in such a system?

What if a team ganged up? What if that team had been hired by a company that profited from the promotion of a link? How do you train a computer to figure that out using algorithms? Back in those days, you didn’t. You couldn’t. AI was not that advanced.

This is why we promoted the Ghost Machine’s search engine as  “people-powered.” It was an intended irony. Our people-powered search engine was to be powered by ghosts. Our ghosts would produce better results than their spiders. Still spiders had the advantage. Automation was essential for growth. So, planning ahead, there were a few things that we could do and were working into our system before operations shut down entirely in 2001-2. The gist of it was rating the ghostsurfers. This would affect an algorithm.

To earn a good reputation, a member would specialize in a category and become a regular contributor. The motive for this was revenue sharing. Users were paid to give feedback on the quality of the search engine. They specialized in the categories they chose. Their peers had the ability to contest their rankings. If they opted in to notifications, they could set the system so that if link positions changed for a category that they claimed to be a specialist in, they would have an opportunity to respond. If feedback on their feedback was given in a forum, they were also notified.  Our forums were quite busy and we also had very active chat rooms.

Ultimately, the reputation of a ghostsurfer could go up or down. They would offer a rank choice for every link in a category, as a category specialist. Their vote, not for a link, but for the whole category, was weighted by their reputation. If they’re recommendations were continually refuted, their reputation would go down. If they were one of a hundred ghostsurfers ranking that category and their reputation was 80/100, their vote might count eighty times, but if their reputation was the lowest possible (0/100), their vote might count zero times. But do you see the problem here? Their friends could vote them up, falsely boost their reputations, and the system could be manipulated.

It’s accountability – a system of checks and balances. We solved the problem with an auditing system. Members would earn ad revenue shares by rating raters, not just by ranking category results, and anything and anyone could be rated. Quality mattered more than quantity. Everyone was accountable. Persuasion on the forums made a difference. Being present where the dialog took place, offering persuasive arguments, was how reputations were improved. No one could suddenly start using the system to manipulate it. They had to earn a reputation if they were to change its results. Advertisements were to the side, clearly marked. Key word searches did not auto-suggest, though there was some toying with how to do so fairly, if we did.

Politics

I don’t intend to use the same search algorithms I was building into the Ghost Machine on the CounterChecker app. It is a very different beast. It may well expand into other content categories assuming its first iteration works, but it will begin where the greatest point of pain currently is – the suppression of political ideas. Right now, there is a bevy of pissed off conservatives who are sorely entrenched in the belief that social and traditional media, and our educational institutions are almost the direct equivalent to the mouthpiece of the Democratic Party. Many even think that the intelligence agencies are feeding it on behalf of a military industrial complex with an agenda. I don’t think I need to elaborate. Suffice it to say, the pain is real.

In another post, I elaborated on my own political biases. We all have them. But we don’t all pursue facts the same way. For me, I think that a good instrument for discerning things is essential. If our instruments skew to the left or the right, it won’t be the truth we are discovering. If we were to use the same algorithms for the CounterChecker app that I was using on the Ghost Machine app, that would be a step in the right direction. But even with user reputations tweaked and checked by rated rating raters, members would try to manipulate results.

Teams

Before we closed operations, we had almost 30,000 ghostsurfers. Of these, about 200 were very active, at most. Anyone could be a ghostsurfer. Each person was on their own. The CounterChecker app will use a team leader process instead. Each team will have a single voice, through that leader. For the CounterChecker app’s first iteration, the leaders will be chosen and assigned by the political party that they are associated with. Therefore, we will start with two fact checking teams – one on the political left and another on the political right. The team leaders will delegate their work to as many “CounterCheckers as they need in order to handle the workload.

Counterchecking is a more time consuming and elaborate process than fact checking. With fact checking, dubious material is refuted with a single sweep. A fact checker can usually get away with a small amount of drive-by journalism, produce a page of information explaining why they think something is wrong, and then they can move on to their next victim. Counterchecking is a more carefully detailed process. The assigned counterchecker is aware that their fact checks will be fact checked by a capable opponent. They will not be directing their work as comfort food to gullible masses. The first to see it will be the opposing team. Their own work will be hyper-linked to the work of their critics. Every single word they say can be directly refuted with an entire Wikipedia-sized refutation by the opposing team’s designated countercheckers.

Research

The result is research. Rather than giving us an entertaining shouting match, as we see sometimes on certain political news shows and limited debates, all viewpoints get as much time and space as they need to make their case. Each team has a chief editor. The team’s chief editor assigns category experts. The chief editors are directly connected to the political party apparatus. They fund their own research. They promote their own ideas. They create their own pages.

We will pay them a portion of the advertising revenues to compensate them for their work. Since the teams are in opposition to one another, the CounterChecker app will be any journalist’s dream for doing research. If there is supporting information to be found that is not already included in a dispute, the researcher can supply it at any time. New journalists who are not counter checkers, but who would like to be, can issue suggestions to CounterChecker team members, directly corresponding with them. If their work proves itself, maybe the Chief editors will hire them.

The result is that the CounterChecker app becomes the definitives source of information for all things political. Chief editors coordinate their team’s information by delegating the workload by category, so that it doesn’t become overwhelming.

Getting Paid

One thing any good chief editor is aware of is monetization. Who are the advertisers? One obvious answer is political campaigns and political action committees and lobby groups. The CounterChecker app and web properties will offer copious and highly targeted advertising space to an interested audience. These advertising dollars will become the source of research funding, and also the source of promotion of the CounterChecker app itself. Once linked throughout the Internet, the CounterChecker app then becomes a household word, like “Google.” Did you Google that? Did you Countercheck that? Same thing. Advertising of the app itself is hardly necessary under those circumstances. But everyone wants to get paid, especially the chief editors.

When YouTube started offering revenue shares, there was a certain gold rush because anyone and everyone could earn some extra money from home by contributing content if they enabled ads. With the CounterChecker app, any journalist will be able to earn money in a similar way. The percentage they earn will be whatever portion the chief editor determines based on the analytics in the CounterChecker app. The chief editor will share in all the revenues of their team and can negotiate the amount as they see fit from their own share. The first iteration will require ad space but later versions will add an option to turn off ads on a page by page basis.

An Illustration

How will this work? You want an illustration. So, let’s say there are 2,000 fact checks in the CounterChecker database – 1,000 generated by the left and 1,000 generated by the right so that the chief editor on the left has created 1,000 fact checks. And let’s say, the average fact check has one hundred branches in its tree of truth. Further, let’s say the average number of page views countercheck page is 1,000 per day so there are 100,000 page views per day per fact check. This means 100 million page views per day per chief editor. This is for illustration purposes. Actual results will vary. But this would mean that the chief editor on the left would be able to split up revenue as they saw fit for 100 million page views. If after the CounterChecker app provider took its share, the average page yielded an additional 1/10th of 1 cent per view for the editing teams, that would be $10,000 in daily revenue for the chief editor on the left to split up. The left’s chief editor could keep $1,000 if they set their portion to 10% of their total network in this example.

Their researchers would be paid the other $9,000 that day. If there were a hundred of them, that would be $90 per counterchecker per day on average, in this example. Again, this is not an indication of actual income. It is strictly for illustration purposes. The money will accrue residually. So the longer a counterchecker stays active, the more articles they will countercheck. As a result, the more ad space they will create and the more their personal analytics will boost their daily income. Obviously, the counterchecker that counterchecks ten articles per day stands to earn more than the counterchecker who counterchecks only one article per week. A chief editor has the right to dismiss an inactive or substandard counterchecker at any time. They select their own team. If the counterchecker wanted to dispute an unfair dismissal, they would have to take that up with their chief editor or their assigns.

Normally, counterchecking will be a freelance opportunity. If a chief editor wants to guarantee a salary or wage to any of their researchers, that is their option. We will want to be as accommodating as possible. However, any pay to an employee would not come through the direct deposit system provided by the CounterChecker app in any of its initial iterations. We will offer a revenue sharing opportunity. We will not become an employment network. My point is that chief editors determine what to pay themselves and what to pay their researchers based on our app and site’s analytics.

Analytics

Analytics is a remarkable thing. Our servers will normally be able to tell not just how many times an article has been seen, but where it was seen from and what sort of user profile that viewer has, whether it was a robot looking at it, someone on a VPN, where they were when they saw it, how much time they spent looking at it and more.

Our ad servers will need this type of information to target with. This makes our ad space more valuable. Let’s say a political candidate wants to advertise to Democrats on our site during a primary season. They don’t want to advertise on our conservative generated pages. They may wish to pay a premium because they are strong on a certain issue and target within a specific category. They will almost always wish to target their specific swing areas. Location analytics are critical in that regard.

Analytics are important for advertisers. They are also important for content creators. Whereas, Wikipedia is a non-profit organization, the CounterChecker app is for profit. When creating content, chief editors will be aware that not every subject is a potential revenue generating subject. We will possibly find ways to broaden the scope of our content to less lucrative areas of interest in later iterations of the app, but initially, countercheckers can rest assured that most pages will be available for revenue generation and that their chief editors will likely have an eye on how best to manage each category to both maximize profit and win their side of the arguments. They need qualified researchers in those categories to make that happen, so it will behoove them to pay them generously. The analytics will provide  a method by which fair pay can be negotiated and distributed within the budget limitations of the overall enterprise as determined by revenues first received.

Searching

The CounterChecker app is not going to be a search engine. It will be a fact finding machine. This means we don’t have to worry about ranking links in categories as I did with the Ghost Machine. Instead, what we will do is make the CounterChecker app hookable to social network platforms, search engines and other apps. Presently, media is being divided into left and right in two biased streams. The CounterChecker app will be the balancing point between them. We aren’t here to replace them, just bring them true independence through a systemic discourse and the convenience of a little code that can be added to any web site or social media platform to punch in a link to any related post a key word might trigger, to any disputed meme, statement or article, as fact checks presently do.

As a fact finding machine, the CounterChecker app will offer search features. Really, it will be a more advanced search system. Once on a CounterChecker page, a reader will be able to enter a key word into a search box. That you’ll be familiar with. But we anticipate our readers will be more astute than average, therefore, we will offer advanced search features.

Search modes begin with the Tree of Truth. Any link to a counterchecker page is a branch along that tree, or its trunk. So there are navigation features built around that tree, designed to help a researcher follow the discourse surrounding any disputed point and to see how many branches follow and what categories they fall under, and which teams provided them.

Either a text has been challenged, or not. If a counterchecker page has not been challenged, it will be presented as a fresh twig. An editor can decide to end a conversation, or add a placeholder, like a bud on a twig. This can later grow into a branch. This will indicate to a reader that a response is forthcoming. They can request a notification once it is published.

Archiving

News cycles come and go and people are generally addicted to the latest news. That’s why news feeds are so popular. We like to hear the very latest thing. A fact finding machine does not work that way. Search engines that place emphasis on news feeds actually bury facts by making yesterday’s news less relevant. And less relevant things tend to not show up in searches. The result is that the past tends to be forgotten.

A Tree of Truth does not forget. When I was developing the Ghost Machine, this was something I considered very deeply. At the time, I was concerned that people and their work was being forgotten. Legacy.com did not exist when I first conceived of the Ghost Machine. Our solute to one another as ghostsurfer members was “Remember!” Life is precious. It should not be forgotten. It also offers many valuable lessons. We build on the past. Archiving is thus, not just some quaint place we can occasionally visit. It is the core launching point to what we are becoming. Like it or not, it is a critical foundation. And it should not be revised by the next generation. It should be accurately preserved, just to be fair. We are neither demons nor angels. We are what we are, living beings, with legacies we can all benefit from.

Neither should our political opinions change with the winds of the latest trends. I say all this to point out that a news feed mentality is actually quite dangerous. It sets us up both for historical revisionism and the failure to learn valuable lessons from history. There will be people to search, and true and false things said about them. And there will be categories of ideas to search. If a user wants to search by date, say those submitted in the last hour in a certain category, that might be a way of catching fresh articles. There are also notification process and preferences for that. But my point is that searching will be oriented differently. You aren’t necessarily looking for new information. You are looking for all of the details related to a certain subject. The interface will be designed to make that as easy to do as possible.

Speed

There is a technical side to this that involves our cost and, importantly to our users, our speed. News feeds serve data quickly by putting commonly accessed images and data into the cache of network servers. It makes the web work faster. It speeds up download times. Less often accessed archived information is slower. End users don’t tolerate slow downloads. Major media sources understand that it is both faster and cheaper to put the most popular images and data closer to the end users. Their bandwidth is reduced this way. Youtube, Google, Facebook and Twitter have all worked with Internet Service Providers to optimize new and old content and to continually prep out fresh news for quick propagation. They even own much of the infrastructure. What do you think they’ve done with all that capital?

This sort of thing is controlled through another sort or algorithm that ties in with analytics. Stale and non-accessed data, takes a back seat position in terms of cache priority. The CounterChecker app will not be a news site any more than it will be a search engine, but it will still advance the most commonly requested data to the random access areas of the web for ready view as news and search sites do. The difference is that since whole tree branches and trees full of information are likely to be accessed, we will have to struggle with speed. Normally, this will mean downloading adjacent and most common pages silently in the background, first to ISPs and then to end users, before they are even requested, while the current page, or portion of a page, is being viewed.

Algorithms

That’s a bit technical. The bottom line is that there are ways to make the app work efficiently and do its job. The whole thing will be run on algorithms. What will make the system fair, will be better algorithms. What will make it achievable, is algorithms distributing highly targetable advertising. What will track revenue distribution, is analytics working together with algorithms. I thought you should know about the CounterChecker app’s algorithms. Popularity will not be what brings users results. Truth will. These will be the algorithms of the Tree of Truth.