Saturday, 7 March 2015

Apprehended bias and pornography

During my appeal to the Full Bench of the Federal Court of Australia this week an interesting question arose regarding apprehended bias which also relates to pornography.

Porn folder (Source XKCD)

In my submissions there were many legs to the apprehended bias allegations and Honourable Justice Beach questioned the nature of many strands leading to apprehended bias. It is a good question. I think the best answer to such a thought comes from Justice Potter Stewart of of the United States Supreme Court in 1964 when regarding pornography he uttered, “...I know it when I see it...

The full quote regarding “hard-core pornography”, which is not to be protected free speech, was famously quoted by Justice Potter Stewart in Jacobellis v. Ohio:
“I shall not today attempt further to define the kinds of material I understand to be embraced within that shorthand description ["hard-core pornography"], and perhaps I could never succeed in intelligibly doing so. But I know it when I see it, and the motion picture involved in this case is not that.”
A similar discretionary notion might be said to apply to apprehended bias from a court's view point. There are similar difficulties in trying to define a hard test but in an instance like the one I was facing there should really be no doubt that apprehended bias existed in the totality of the combination of circumstances.

Just as it may be appropriate if you find a $1 coin on the pavement to not feel guilt in blessing your luck and putting it in your pocket; it would be inappropriate to not turn to the authorities when finding a thousand $1 coins on the street. At some point such coin collection crosses a threshold of community standard of relevance. A similar notion exists in the old many versioned morality story of a lady being willing to compromise her morality for a large sum and being offended with a small sum. It usually ends with,
"W: What kind of woman do you think I am?
M: I thought we'd already established that. Now we're just haggling over price."
In this way, apprehended bias is indeed like pornography, although the balance of probabilities should be biased differently. In both cases caution should be the consideration.  For pornography caution means the obscenity should have clarity beyond camera focus. For apprehended bias the concern is impartiality. The balance of consideration must be that impartiality is not to be risked for public confidence in the judicial process.  This implies a different focus on the grey line. Pornography should be without doubt obvious to meet the consideration of guilt. Apprehended bias should be seen to be a likely risk in the reasonable perception of the mythical fair minded layperson. There is an awkward tension here as the perception of impartiality is not be risked but a conclusion of apprehended bias should not be too easily reached.

Thus, “I know it when I see it” seems to be a good test for both apprehended bias and pornography.

Thursday, 22 January 2015

Youtube clip: Speed Demonized: High Frequency Trading Helps, Not Hurts Investors

This is a pretty funny video on HFT I enjoyed. Not sure I agree with all the commentary but the sentiment is heading in the right direction. One thing HFT is not good at: communicating the message. Efforts like this, perhaps, help.

It makes a claim that retail is dumb money. I don't really think that is true. The wisdom of the retail crowd is often smarter than your average bear. It misses the point that HFTs need to make money every day otherwise they are just doing it wrong and their business will fail.

It also unfairly critiques institutions as investors a little. Those institutions ultimately exist to serve both their customers and shareholders which includes retail clients. Nevertheless, it is funny.

Happy trading,


Thursday, 15 January 2015

Mellanox Connect-X 4 details emerge: 700ns

Extract from today's Linley Wire

Mellanox Brings 100Gbps to Servers 

By Loring Wirbel

Mellanox recently introduced single- and dual-port ConnectX-4 host channel adapters that support InfiniBand (IB) and Ethernet protocols to 100Gbps. They are based on the company’s Virtual Protocol Interconnect technology for converged IB/Ethernet environments. The ConnectX-4 package also includes advanced software for configurable networking hierarchies, including RDMA over Converged Ethernet (RoCE, pronounced “rocky”) v1 and v2 and the new InfiniBand Dynamically Connected Transport (DCT).

The card can process 150 million messages per second with typical latencies of 0.7 microseconds. ConnectX-4 is intended to serve in any IB or Ethernet hierarchy, handling speeds of 10Gbps, 20Gbps, 25Gbps, 40Gbps, 50Gbps, 56Gbps, and 100Gbps. The card’s main applications include I/O Virtualization and virtualized overlay networks based on such encapsulations as VXLAN, NVGRE, Geneve, and MPLS. The single-chip ConnectX-4 VPI and ConnectX-4 EN controller ICs launched simultaneously with the PCIe card, offering a mix of 100Gbps and lower-speed connectivity, all based on a PCIe Gen3 x16 host interface.

Although ConnectX-4 is a natural follow-on to ConnectIB (IB only) and ConnectX-3 Pro (multiprotocol) for 56Gbps FDR, Mellanox’s transition strategy for Ethernet customers still must take into account existing users of 10Gbps and 40Gbps products. The company says it is succeeding in 40Gbps Ethernet adapter markets, but it also emphasizes the enhanced L3 routing capabilities and greater performance that ConnectX-4 gains from the mix of 100Gbps ports and RoCE v2 enhancements.

Networking Report subscribers can access the full article here


25 February 2015: New performance claims for Connect-X4

" throughput of 195Gb/s, applications latency level of 610 nanoseconds and message rate of 149.5 million messages per second, ConnectX-4 is the highest performing adapter for the HPC, Web 2.0, cloud, machine learning, storage and enterprise applications." from HPC Wire

Saturday, 13 December 2014

Sec. 309. Procedures for the retention of incidentally acquired communications

The only thing necessary for the triumph of evil is that good men do nothing.

H. R. 4681 Intelligence Authorization Act for Fiscal Year 2015.

Passed with 325 votes to 100.

Yep, it's pretty onerous and Orwellian, as is usual. It has slipped though Congress with scant attention and virtually no debate.

Section 309 is the most damning element. It contains enough outs and weasel words to allow the US intelligence community continue to break the US constitution forever. As mentioned, it is most troubling that this Act, with such a terrible Section, seemed to slip through Congress without too many people paying attention to the detail. Where was the debate?

The text is reproduced below, but here are a couple of highlights:
(1) Covered communication 
The term covered communication means any nonpublic telephone or electronic communication acquired without the consent of a person who is a party to the communication, including communications in electronic storage. 
A covered communication shall not be retained in excess of 5 years, unless— 
(iii) the communication is enciphered or reasonably believed to have a secret meaning; 
(vi) retention is necessary for technical assurance or compliance purposes... 
(vii) retention for a period in excess of 5 years is approved by the head of the element of the intelligence community responsible for such retention...
So, what do they do if they want to keep stuff for as long as they like? They can keep it forever by simply approving the retention themselves using (vii) above. They could keep anything just for technical assurance or for their own deemed compliance reasons (vi). Also, (iii) could easily be misinterpreted to provide their own basis for whatever they wish to do.

Of course, if you're one of the planet's 6.7 billion non-US citizens, there is no change. Your data continues to be owned by the greatest back-up and archival facility ever known to man:
(iv) all parties to the communication are reasonably believed to be non-United States persons
What kind of parliament would let something like this slip through with virtually no debate? Distracted by Christmas, budget approvals and the torture debate? Quite clever of the intelligence community to manage the political agenda with enough clout to get this through. It reinforces the idea that security is a game of unrelenting persistence, patience and scheming. Well done NSA and friends.

Eisenhower would be turning in his grave.


PS: A late burst from Congress managed to get 100 people to provide futile resistance but it was too little, too late. The NO vote was split fairly evenly between Democrats and Republicans.

PPS: Tim Berners-Lee on why internet freedom is important.


Sec. 309. Procedures for the retention of incidentally acquired communications

In this section:
Covered communication
The term covered communication means any nonpublic telephone or electronic communication acquired without the consent of a person who is a party to the communication, including communications in electronic storage.
Head of an element of the intelligence community
The term head of an element of the intelligence community means, as appropriate—
the head of an element of the intelligence community; or
the head of the department or agency containing such element.
United States person
The term United States person has the meaning given that term in section 101 of the Foreign Intelligence Surveillance Act of 1978 (50 U.S.C. 1801).

Procedures for covered communications
Requirement to adopt
Not later than 2 years after the date of the enactment of this Act each head of an element of the intelligence community shall adopt procedures approved by the Attorney General for such element that ensure compliance with the requirements of paragraph (3).
Coordination and approval
The procedures required by paragraph (1) shall be—
prepared in coordination with the Director of National Intelligence; and
approved by the Attorney General prior to issuance.
The procedures required by paragraph (1) shall apply to any intelligence collection activity not otherwise authorized by court order (including an order or certification issued by a court established under subsection (a) or (b) of section 103 of the Foreign Intelligence Surveillance Act of 1978 (50 U.S.C. 1803)), subpoena, or similar legal process that is reasonably anticipated to result in the acquisition of a covered communication to or from a United States person and shall permit the acquisition, retention, and dissemination of covered communications subject to the limitation in subparagraph (B).
Limitation on retention
A covered communication shall not be retained in excess of 5 years, unless—
the communication has been affirmatively determined, in whole or in part, to constitute foreign intelligence or counterintelligence or is necessary to understand or assess foreign intelligence or counterintelligence;
the communication is reasonably believed to constitute evidence of a crime and is retained by a law enforcement agency;
the communication is enciphered or reasonably believed to have a secret meaning;
all parties to the communication are reasonably believed to be non-United States persons;
retention is necessary to protect against an imminent threat to human life, in which case both the nature of the threat and the information to be retained shall be reported to the congressional intelligence committees not later than 30 days after the date such retention is extended under this clause;
retention is necessary for technical assurance or compliance purposes, including a court order or discovery obligation, in which case access to information retained for technical assurance or compliance purposes shall be reported to the congressional intelligence committees on an annual basis; or
retention for a period in excess of 5 years is approved by the head of the element of the intelligence community responsible for such retention, based on a determination that retention is necessary to protect the national security of the United States, in which case the head of such element shall provide to the congressional intelligence committees a written certification describing—
the reasons extended retention is necessary to protect the national security of the United States;
the duration for which the head of the element is authorizing retention;
the particular information to be retained; and
the measures the element of the intelligence community is taking to protect the privacy interests of United States persons or persons located inside the United States.

Sunday, 7 December 2014

FINRA ATS NSM Tier 1 weekly stats - week ended 10 Nov - updated 1 Dec

IEX is now #4. That Lava Flow at #6 is interesting in the context of their decision to shutdown in 2015. Credit Suisse and UBS remain at #1 and #2.

ITG is #12. One has to wonder if ITG customers know that "agency only" means that a customer's order for some names could possibly be traded against the ITG proprietary trading book which is hidden in plain sight? At least with Credit Suisse and Goldman Sachs you know of their potential to be on the other side of your trade. They don't mislead. Sometimes ITG discloses their prop trading activity in their 10-Q's, such as,
"Interlisted arbitrage trading, which accounts for most of our other revenues, generated $1.8 million in the third quarter of 2009, significantly down from the $4.4 million achieved in the comparable 2008 quarter due to a decrease in Canadian and U.S. interlisted trading volumes and an increase in competition in this area." Source: ITG - 10-Q for the fiscal period ended September 30, 2009
That, to me at least, seems to blur the common understanding of "agency only." Does this Bloomberg story hint that ITG's reputation is really as an agency-only shop without prop? I wonder what the SEC and a certain NY AG thinks? Maybe surprised customers could claim their trade fees from ITG back for the last decade or two?

Seven ATS platforms have "per trade" sizes well in excess of 1,000 shares and IEX isn't one of them. That is, despite their #FlashBoys hope to be a platform suitable for large institutional blocks, you still need to use HFT techniques to trade IEX well. IEX seem to be doing a good job but it is perhaps a little different to what Michael Lewis envisaged.

Happy trading,


Name Shares
Per trade
CROS CREDIT SUISSE SECURITIES (USA) LLC 332,102,591 11.25% 1,799,961 12.67% 185
UBSA UBS SECURITIES LLC 327,029,127 11.08% 2,299,498 16.18% 142
DBAX DEUTSCHE BANK SECURITIES INC. 195,254,900 6.61% 1,108,814 7.80% 176
IEXG IEX SERVICES LLC 191,720,074 6.50% 678,625 4.78% 283
MSPL MORGAN STANLEY & CO. LLC 171,558,600 5.81% 777,522 5.47% 221
FLOW LAVAFLOW, INC. 165,601,276 5.61% 834,201 5.87% 199
JPMX J.P. MORGAN SECURITIES LLC 163,801,415 5.55% 741,248 5.22% 221
SGMA GOLDMAN SACHS EXECUTION & CLEARING, L.P. 155,152,765 5.26% 839,811 5.91% 185
MLIX MERRILL LYNCH, PIERCE, FENNER & SMITH INCORPORATED 154,230,349 5.22% 672,986 4.74% 229
BIDS BIDS TRADING L.P. 130,247,400 4.41% 365,785 2.57% 356
KCGM KCG AMERICAS LLC 121,968,993 4.13% 709,040 4.99% 172
ITGP ITG INC. 121,452,700 4.11% 440,372 3.10% 276
EBXL LEVEL ATS 96,458,000 3.27% 616,258 4.34% 157
LATS BARCLAYS CAPITAL INC. 91,596,213 3.10% 551,491 3.88% 166
ICBX INSTINET, LLC 81,872,226 2.77% 338,540 2.38% 242
APOG CITADEL SECURITIES LLC 44,677,617 1.51% 246,816 1.74% 181
DLTA DEALERWEB INC. 43,896,832 1.49% 85 0.00% 516,433
XSTM NATIONAL FINANCIAL SERVICES LLC 42,367,701 1.44% 116,281 0.82% 364
LTPL CREDIT SUISSE SECURITIES (USA) LLC 38,513,986 1.30% 246,434 1.73% 156
PDQM PDQ ATS, INC. 37,457,924 1.27% 166,735 1.17% 225
MSTX MORGAN STANLEY & CO. LLC 34,003,400 1.15% 201,219 1.42% 169
LQNT LIQUIDNET, INC. 31,477,100 1.07% 676 0.00% 46,564
IATS INTERACTIVE BROKERS LLC 29,974,679 1.02% 95,669 0.67% 313
PULX STATE STREET GLOBAL MARKETS, LLC 25,872,846 0.88% 2,063 0.01% 12,541
NYFX CONVERGEX EXECUTION SOLUTIONS LLC 25,629,175 0.87% 38,386 0.27% 668
CXCX CITIGROUP GLOBAL MARKETS INC. 24,109,306 0.82% 140,051 0.99% 172
VRTX CONVERGEX EXECUTION SOLUTIONS LLC 17,684,595 0.60% 76,793 0.54% 230
LQFI CITIGROUP GLOBAL MARKETS INC. 13,384,483 0.45% 163 0.00% 82,113
XIST INSTINET, LLC 13,256,200 0.45% 18,602 0.13% 713
BOOK BLOOMBERG TRADEBOOK LLC 9,380,298 0.32% 41,439 0.29% 226
LQNA LIQUIDNET, INC. 7,942,400 0.27% 3,634 0.03% 2,186
MSRP MORGAN STANLEY & CO. LLC 2,499,700 0.08% 8,015 0.06% 312
AQUA AQUA SECURITIES L.P. 1,700,850 0.06% 181 0.00% 9,397
LEHM BARCLAYS CAPITAL INC. 1,438,933 0.05% 29 0.00% 49,618
RCSL RIVER CROSS SECURITES, LLLP 1,333,732 0.05% 5,581 0.04% 239
JEFX JEFFERIES EXECUTION SERVICES, INC. 706,586 0.02% 2,888 0.02% 245
FNBR FOLIOFN INVESTMENTS, INC. 425,410 0.01% 11,067 0.08% 38
WDNX WEEDEN & CO.L.P. 110,419 0.00% 167 0.00% 661
PROS PRO SECURITIES, L.L.C. 100 0.00% 1 0.00% 100

Total 2,951,782,201 100.00% 14,209,505 100.00% 208

FINRA ATS stats from June 2014 FWIW.
Original FINRA data

Sunday, 30 November 2014

Ode to Teaching

It's another end to a school year. For months, the world over, an army of teachers have been toiling in the confines of the education factories leading to a final preparatory push and some exhaustion.

There is often much personal and career sacrifice that goes with the decision to be a teacher, and, to continue as a teacher. It is not a profession renown for its pay, chefs on duty, foosball tables in the break out room, nor, sadly, community respect. Stock options are fairly rare. Dedicated professionals are haunted by the cruel and excruciating epitaph echoed by far too many, "Can't do, teach."

A warm thank you from their students sometimes makes all the difference.

A lovely reward for a year's work.
(click to enlarge - broken out below)

As many teachers do, at year end my wife appreciatively received some small gifts and kind words. Carolyn often pecks on her keyboard late into the night. Rises early for gym, the cross town commute, and the 7:30am "extra" tutes. Every day is a pretty long day. The occasional aggressive and threatening parent at a parent-teacher interview challenges the spirit. Each year, every semester, every class, is an opportunity of relentless importance. The wheels never stop. I'm proud of my caring wife and her choice to be a teacher. It been a tough couple of years with twenty years of her teaching superannuation being stolen, a cancer scare, suicide attempts and subsequent hospitalisation in the direct family. Some days aren't easy. Sometimes a note of thanks, like this one, makes that life choice of being a teacher completely worthwhile. 

Hopefully my wife doesn't see this too quickly so that I don't get told to take this note down. She's not one for a fuss or embarrassment. Yes, this card is a fabulous reflection on my wife, but it also shines the same light on the dedication of all her fellow professionals. When we read the unadulterated thoughts of appreciative students, we can all appreciate the importance and fine work that too often is hidden in the classroom. It's a wonderful profession for which we should all give thanks.

I hope this is a nice reminder that in the lives of our children and in the progression and advancement of our society:

Teaching Matters

A respectful token to energize the soul
(click to enlarge - broken out below)

One card for one soul can provide more heat than a fistful of dollars. 

If you know a teacher, take a moment and say a simple thanks. A little kindness can travel far.

Sunday, 23 November 2014

SC14 view from afar

Supercomputing 14 in New Orleans has wrapped up. Despite the stasis in the top 500 echelons some meanderings of interest to traders continue to emerge from HPC.

DE Shaw's Anton 2 - click to enlarge
(The Anton 2 Chip: A 2nd Generation ASIC for Molecular Dynamics Simulation Hot Chips 2014)

The BIG announcement for me for the week was Xilinx's SDAccel. Every latency focused trading firm is going to have to evaluate this framework from a functionality and productivity point of view. There have been quite a few different toolkits for doing this kind of work over the years but this feels a game changer.

I've previously been involved with Impulse-C and found it an interesting and useful product from a good company. However, it was largely not used in my shop as the types and pragmas required in the code essentially removed the C'ness which made it yet another C-like foreign language in practical terms. We found it more productive to just go straight to VHDL code at the end of the day, so the Impulse-C, though impressive, became shelf-ware. Perhaps if we had more of a computational burden it may have been more useful, but much of the work at the time was tricky I/O and thus VHDL mattered more.

The nicest implementation of C-like hardware languages I've reviewed over the years was Handel-C. Lovely CSP / Occam / Pi-calculus like constructs. Quite elegant code. Not C though, but this was not that important for me at the time. The killer was that the platform implementation targets were limited and the performance of the code was somewhat, er, um, challenged. Limited performance is not what you're looking for when biting off hardware implementations. Impulse-C targeted VHDL or Verilog so it had the advantage that the specific platform compiler could optimize. Handel-C tried to do more and targeted the RTL level directly but it didn't quite do the terrific job the native compilers could do. I think they lost an opportunity there.

There have been a bunch of others over the years. For example, Mitrion-C was bundled with Cray's hybrid FPGA supercompute platform for a while (Cray XD1(TM) supercomputer) and SGI's RASC. No stand outs for success. C-like programming for FPGA platforms is hard.

Hopefully, Xilinx's SDAccel can pick up some success. I think the big game changer is both the support and weight of Xilinx behind the platform and the availability of partial reconfiguration on the fly. It remains to be seen whether the intrusions of specific types, code styles and the dreaded pragmas overwhelms the benefits a high level language brings, but it sounds quite the advance to me. I'm optimistic. An impressive aspect from the demos was the occasional improvement possible over raw VHDL from both clock and area perspectives. When combined with SDNet and their QPI toolkit, extremely compelling architectures may be possible with small development teams. Let's hope the C-like approach works for FPGAs this time. All trading firms will need to have a look at this kind of framework if truly low latency is what makes them tick.

Some other tech from SC14

Intel had some interesting announcements, not all at SC14. The further development of the Knights platform to Knights Hill is very compelling, especially when combined with HMCC memory. With the progression from Corner, Landing to Hill, there is plenty of geographical growth possible. I can't wait to see what the Mountain platform or 2030 Galaxy platform looks like ;-)

The release of HMCC 2.0 and the support of various vendors, including Xilinx, is exciting for a few reasons, but the main one for me is the indication of continued growth in 3D, or 2D stacking, manufacturing. Intel was also talking about this for Flash memory, just as Samsung has recently done, and Intel presaged packaging solutions with around 10TB of Flash in SSDs in only a couple of years. NVidia also previously announced stacking of memory with their Pascal GPU platform. NVidia's Volta, announced for Summit and Sierra, will continue that. Continued development of innovative packing will drive densities higher and costs down though at a cost of accessibility as it seems this is a bit like the new PCB but only accessible to the very few.

D E Shaw's Anton 2 chip was an exciting platform to read about. I'm not sure if it won the Bell prize, but if it didn't, I think it probably should have (* update: it did). One notable takeaway from reading their paper is that so much more is still possible with further overlap, out of order speculation and processes beyond 40nm and 2B gates. I can't wait to see Anton 3. It is good to see a firm be prepared to build a custom ASIC. Their last one, the original Anton, was done by an e-beam house in Japan but the markings on this chip are from Korea, so I wonder who they are using to fab there? Whilst it is for molecular dynamics, I'm sure every trader would love to have such awesome power at their finger tips for "Hardware Support for Fine-Grained Event-Driven Computation." Though I'm sure their research motives are altruistic, I'm also sure a clever trading firm would find a use for that ;-)

100Gb connectivity continues to mature with announcements from Mellanox of a sub-90ns 36 port 100Gbps IB switch and with the Invea-tech platform, previously mentioned, being demonstrated at SC14. I must say I find the 25G Ethernet movement, also on demo at SC14 running "under 100ns" platform to platform, a little more practical as a future path for now.

Mellanox, in addition to their 100Gbps IB/Eth card, also joined SolarFlare in having an FPGA NIC solution. It is an alluring solution that looks like it may be better than the SolarFlare for latency as it supports FPGA fabric on either or both of the network and PCIe facing sides. The press doesn't declare the flavour of FPGA nor the tools available. We'll have to wait to understand it better to see if it can compete with a vanilla network connected FPGA NIC for low latency trading.

The Power8 CAPI platform had its first official platform release with Nallatech at the show. A good latency reduction technique to be aware of. I think I'd be focusing on the Xilinx / Intel QPI solution myself though.

Some of my old friends at Metamako were at SC14 next to the Xilinx booth with their compelling platform. If you want the fastest way to get two packets from two wires onto the one wire at 10G, say for facing an exchange, risk gateway or other financial choke point, such as a mandated firewall, then their new MetaMux 32 reigns supreme. It is the fastest way at ~100ns to get two packets onto the same wire at 1G / 10G speeds. This is in additional to their fancy-smancy layer 1 reconfig tricks. I think the biggest use case for me for Metamako is simply using their MetaConnect platform just for timestamping though. You could save a bunch of money using one of their MetaConnects, or perhaps even the MetaMux, to timestamp and collect packet information rather than using expensive timestamping cards but I'm not sure people realise how cool that is from a non-intrusive tap reconfig and cost saving point of view. I'd really like a MetaConnect to script up for dynamic network reconfig and running performance unit tests for financial apps in the test lab.

A mystery for me is the intriguing Intel Omni-Scale platform. Just enough details have been released to know that it is going to be interesting but not enough to really know what it is all about. So, keep your eye out for developments there. Intel have long had the opportunity to kill off much of the NPU, custom processor and even perhaps the entire NIC market by incorporating compelling networking interfaces into the coherent level of their microprocessors. It is not clear when and how this may happen with Omni-Scale. Vendors relying on NIC card revenue should be starting to shift a little uncomfortably in their seats. A big leap in network latency reduction could be at hand but there remains an opportunity for a firm like Cavium or AMD to gazump Intel on the low latency network and compute combination. Direct fabric integration, not the kludgy PCIe SoC kind, is long overdue for mainstream processors. Maybe this is at last on its way.

Perhaps not so relevant to trading apps, the announcement of Summit and Sierra was intriguing with their reliance on nVidia's Volta with stacked RAM and NVLink. It is certainly a nice win for the IBM Power 9 platform even if the headline performance is really reliant on nVidia rather than the Power 9. It also just shows how hard an ExaFLOP is, at least for LinPack, as both of don't quite get to ExaFLOP performance, so the ExaFLOP race remains on.

Happy trading,


{NB: Scant public details from Intel on Knights Hill (optical connect, stacked memory, faster) and Omni-Scale/Path.}