Monday, 17 November 2014

Alpha Data - PCIe 3.0 Xilinx UltraScale 10/40G

I was browsing the exhibitors at SC14 which is on this week in New Orleans. I noticed that I had missed this nice looking Xilinx Kintex UltraScale ADM-PCIE-KU3 from Alpha Data which was news way back in April 2014.
Source: Alpha Data - Data sheet for ADM-PCIE-KU3
(click to enlarge)
Kintex UltraScale is now going mainstream so perhaps just noticing it nowish is cutely serendipitous. I'm not sure about the position of the SMA connector for timing as it looks a little inconvenient in this picture though the picture was an early release. Hopefully it has moved. PCIe 3.0 allowing direct access to Intel uP cache memory (data direct), from Sandy Bridge onwards, would allow lower latency than competing PCIe 2.X solutions. PCIe 2 solutions really should be avoided in this day and age if you're going to the trouble of eeking out the little bits of latency you can.

Simple is good. It is indeed a pretty simple board with up to 8 x 10G lines via break out cables or 2 x 40G, plus additional RAM. Not sure too many in trading would find the SATA ports useful but some may for direct recording to disk but I can't imagine too many bothering.

It looks a good choice and worth checking out at their booth at SC14 this week if you're after a pretty clean Kintex UltraScale design for PCIe 3.0 goodness.

I still think I'd prefer a Zynq solution, such as the Hitech Gobal board HTG-Z7-PCIE-HH or one of the not yet released Arria 10 ARM SoC based boards (e.g. BittWare’s A10PHQ). Still, I'd think a Zynq 7045 would be preferable to the 7100 on the HTG as I'd feel better trading better transceivers for less FPGA fabric, so I'm yet to find the goldilocks board for me. I'm interesting in hearing about boards that you might have found that are just right for your network oriented application. Drop me a mail.

Happy trading,

--Matt.

_____________
PS: Nearly two year old FPGA summary from this blog
PPS: Terasic FPGA board previously mentioned


Saturday, 8 November 2014

Don't buy an ExaNIC from ExaBlaze

I've seen more dubious PR from the nasty people at Zomojo/ExaBlaze recently.

I thought it worthwhile warning people once more before they get sucked into ExaBlaze's reality distortion field.

Specifically: why their ExaNIC cards are not a smart idea; and, why you should steer well away from such wasteful expenditure.

It would be easy to be seduced by the pitch for the ExaNIC but the pitch is hollow with parts simply straight out lies. It's not your fault if you've been deceived into purchasing one.

There are two potential reasons for looking at an ExaNIC:
   1) A low latency network card, and
   2) An even lower latency FPGA trading solution.

Neither make sense. Let's meander through why.

Is it a good NIC?

In the first case, the ExaNIC card is feature poor and it has little advantage over a much better solution from Mellanox or SolarFlare. Mellanox[2] and SolarFlare[1][3] are quality NICs that have rich feature sets, kernel bypass and trusted implementations with open source drivers. Also, Chelsio and Myricom are worthy of some consideration if you have specific features you need.

You should be concerned about the signal integrity of the cards from ExaBlaze as the initial versions had poor to little signal integrity on the networking side which is a difficult problem to spot and solve. It's fine when it works but potentially devastating when you're having to solve problems. ExaBlaze push silly claims about latencies quoting 60 byte packets when the minimum frame length on 1G and 10G is 64 bytes. A 60 byte frame is illegal and referred to as a "runt" frame. To top it all off, the ExaNIC cards are expensive. Sure, buy one to play with if you must, but it makes no sense to invest in the card as a NIC solution.

If a quantum of a hundred nanoseconds is really important to you then, regardless of any NIC, you should be considering an FPGA solution to avoid the PCIe latency cost.

So, what about the ExaNIC as an FPGA solution for trading?

Just don't.

As an FPGA solution it is a much simpler and clearer case to weigh up. It is one of the slowest 10G or 1G Xilinx FPGA solutions you could possibly use. Yes, one of the slowest. If you are going to go all the way to an FPGA solution, which is an expensive way to go, then make sure you do it right.

The fundamental limiting factor in the ExaNIC x4 is that the network facing SERDES are off chip and slower than using native on-chip Xilinx transceivers. ExaNIC as an FPGA path is a dumb solution for low-latency. There are plenty of other excellent FPGA boards out there so you have no reason to waste your money. If you have been fooled into purchasing an ExaNIC on the mistaken belief that it was a fast way to connect to the market and trade then you may be entitled to your money back. They are lying to their customers about their high-latency architecture by referring to it as low-latency.

"It hung in the air in exactly the way that bricks don't."

Look at the following picture from the ExaBlaze web site:

(click to enlarge)
The standard way an FPGA NIC card works, except an ExaNIC doesn't. The SERDES for ExaNIC x4 are not on the FPGA, they are external, an extra hop, and higher latency. Don't be deceived.
The diagram clearly shows the network SERDES to be on-chip with respect to the FPGA. This is a lie for the ExaNIC x4 and it may also be a lie for the ExaNIC x2 if it is a similar design. The SERDES on the ExaNIC x4 is an external mux-demux chip and a slower solution than an on-chip FPGA transceiver. Why would a vendor use a higher latency architecture? To use a cheaper FPGA and save money. Don't be fooled. If you have fallen for the ExaBlaze misrepresentation, go get your money back is all I can suggest.

In summary, as NICs go, there is better quality, functionality and more reasonable prices from Mellanox and SolarFlare NICs, so avoid ExaNIC. As an FPGA solution, it is just a dumb idea to invest in an architecturally slow solution. You can't make it fast. It is slow by design. Get a real FPGA board from a trustworthy vendor as there are plenty of good alternatives. The combined ARM & FPGA SoC solutions as NICs are my favourites at the moment even though the ARMs are a little underwhelming. The SoC FPGA fabric to uP hop beats PCIe latencies for interesting hybrid solutions.

You should try to buy equipment from a trustworthy vendor. Matt Chapman annoyed a lot of people with his little video at Inside HPC where he dances around a product that Zeptonics developed and he claimed as his own. What a schmuck. Greg Robinson used stolen IP from ITG when he was sacked as CEO of ITG Australia to compete on the trading front at Zomojo with ITG's prop trading (Canada / NY interlisted arb specifically) and told multiple staff (so it's not just my claim) not to talk to Bill Burdett about it as Bill was still on the Board of directors of NYSE listed ITG Inc. Bill is a Zomojo beneficial shareholder. Then you have my little continuing dispute with them where I hope for a miracle one day that will see my faith in justice restored with some kind of judicial remedy that passes the smell test. Exablaze / Zomojo lied to either the court or their customers claiming Chapman was no longer associate with the trading at Zomojo to customers and yet at the same time claiming to the court he was an officer of Zomojo. Those statements are mutually exclusive, so both couldn't be true. Zomojo/Exablaze lied to the court many times. Zomojo and Exablaze are nasty people. Do you want to buy from a vendor that not only trades against you but misleads you? Caveat emptor.

If you've bought an ExaNIC, you should use their misleading representations as a reason to return the NICs and get your money back. Fortunately for you, there are plenty of good people in the industry making better product. It's simple. Exablaze serves no useful purpose. Use better products.

Happy trading,

--Matt.

______

[1] Some Solarflare 10GbE Server Adapter Features

• LSO, LRO, GSO large packet offloads • TCP/UDP/IP checksums offloads • Line rate packet
filtering • Receive Side Scaling (RSS) • Accelerated Receive Flow Steering (RFS) • NetQueue,
VMQ, SR-IOV • 256 Multicast filters • Jumbo Frames (9KB) • 4096 VLANs/port • PXE boot,
iSCSI boot • IEEE 802.3ae – 10 Gigabit Ethernet • IEEE 802.3an – 10GBASE-T • IEEE
802.3ad – Link Aggregation and Failover • IEEE 802.1Q, 802.1p – VLAN tags, priority • IEEE
802.3x – Pause

Operating Systems: RHEL 5, 6, 7; MRG; SLES 10, 11; SLERT; other Linux; Windows Server 2003, 2003 R2, 2008, 2008 R2, 2012, 2012 R2; OS X v10.6, 10.7, 10.8, 10.9; Solaris x86 10, 11; ESX 3.5, 4.x, ESXi 5.x; KVM; Windows Hyper-V; XenServer 5.x, 6.0.All server adapters support: SR-IOV, 127 VFs per port,1024 vNICs per port.

[2] Some Mellanox NIC features

ETHERNET
– IEEE Std 802.3ae 10 Gigabit Ethernet
– IEEE Std 802.3ba 40 Gigabit Ethernet
– IEEE Std 802.3ad Link Aggregation
– IEEE Std 802.3az Energy Efficient Ethernet
– IEEE Std 802.1Q, .1P VLAN tags and priority
– IEEE Std 802.1Qau Congestion Notification
– IEEE Std 802.1Qbg
– IEEE P802.1Qaz D0.2 ETS
– IEEE P802.1Qbb D1.0 Priority-based Flow
Control
– IEEE 1588v2
– Jumbo frame support (9600B)
OVERLAY NETWORKS
– VXLAN and NVGRE - A Framework for
Overlaying Virtualized Layer 2 Networks over
Layer 3 Networks. Network Virtualization
hardware offload engines 
HARDWARE-BASED I/O VIRTUALIZATION
– Single Root IOV
– Address translation and protection
– Dedicated adapter resources
– Multiple queues per virtual machine
– Enhanced QoS for vNICs
– VMware NetQueue support
ADDITIONAL CPU OFFLOADS
– RDMA over Converged Ethernet
– TCP/UDP/IP stateless offload
– Intelligent interrupt coalescence
FLEXBOOT™ TECHNOLOGY
– Remote boot over Ethernet
– Remote boot over iSCSI
PROTOCOL SUPPORT
– Open MPI, OSU MVAPICH, Intel MPI, MS
– MPI, Platform MPI
– TCP/UDP
– iSER, NFS RDMA
– uDAPL
PCI EXPRESS INTERFACE
– PCIe Base 3.0 compliant, 1.1 and 2.0 compatible
– 2.5, 5.0, or 8.0GT/s link rate x8
– Auto-negotiates to x8, x4, x2, or x1
– Support for MSI/MSI-X mechanisms
CONNECTIVITY
– Interoperable with 10/40GbE Ethernet switches.
Interoperable with 56GbE Mellanox Switches.
– Passive copper cable with ESD protection
– Powered connectors for optical and active cable
support
– QSFP to SFP+ connectivity through QSA module
OPERATING SYSTEMS/DISTRIBUTIONS
– Citrix XenServer 6.1
– RHEL/CentOS 5.X and 6.X, Novell SLES10 SP4;
SLES11 SP1 , SLES 11 SP2, OEL, Fedora 14,15,17,
Ubuntu 12.04
– Windows Server 2008/2012/2012 R2
– FreeBSD
– OpenFabrics Enterprise Distribution (OFED)
– OpenFabrics Windows Distribution (WinOF)
– VMware ESXi 4.x and 5.x

[3] Some specific SolarFlare features

Product Number
SFN7322F
Standards & Compliance
IEEE 1588 v2
IEEE 802.3ae
IEEE 802.3ad
IEEE 802.1Q
IEEE 802.1p
IEEE 802.3x
RoHS Compliant
Power
5.9W (typical)

Precision Packet Time Stamping
7.5ns resolution
Stable Precision Oscillator
Stratum 3 compliant; short term drift < 3.7*10-7 in 24 hours
Server Clock Synchronization Accuracy
Sub 200ns
1PPS-input circuit
Rising edge active, TTL into 50Ω
1PPS-output circuit
Rising edge on-time, TTL into 50Ω
I/O Virtualization
2048 guest OS protected vNICs; 240 virtual functions;
16 physical functions
PCI Express
PCIe 3.0 x8 @ 8.0 GT/s
SFC9120 10G Ethernet Controller
Supports high-performance 10GbE
SFP+ Support
Supports optical & copper SFP/SFP+ modules; Direct-Attach,
Fiber (10G or 1G), 1G/10G combo
1000BASE-T SFP Support
Supports 1G 1000BASE-T SFP modules
Low Latency
Cut-through architecture/intelligent interrupt coalescing
Receive Side Scaling (RSS)
Distributes IPv4, IPv6 loads across all CPU cores;
MSI-X minimizes interrupt overhead
Hardware Offloads
TSO, LRO, GSO; IPv4/IPv6; TCP, UDP checksums
Adapter Teaming/Link Aggregation
LACP for redundant links & increased bandwidth
(compatible with MLAG)
Jumbo Frames
9216 byte MTU for performance
Enhanced Tuning
Adaptive interrupt moderation
IP Flow Filtering
Hardware directs packets based on IP, TCP, UDP headers
Advanced Packet Filtering
4096 multicast filters; 4096 VLANs/port; adaptive TCP/UDP/IP,
MAC, VLAN, RSS, RFS filtering; Accelerated Receive Flow
Steering (RFS)
Intel QuickDataTM
Uses host DMA engines to accelerate I/O
Remote Boot
PXE, iSCSI boot; unattended installation
Management
SNMP, ACPI v3.0
Virtualization Support
VMware ESXi; Microsoft Hyper-V; XenServer; Linux KVM;
SR-IOV
Operating Systems
RHEL 5, 6, 7, MRG; SLES 10, 11, SLERT; other Linux;
Windows Server 2008 R2, 2012, 2012 R2



Wednesday, 1 October 2014

Zomojo / Exablaze's appeal comment

I'd like to thank Zomojo or Exablaze for their legal analysis of my appeal at this link.

If you're perhaps thinking there may be something interesting about trading or tech in this post, please stop reading now. Insomniacs need only apply. It is probably only tangentially interesting to those who have been following the three year old Zomojo/Exablaze v Zeptonics saga.

Without getting too carried away, I must agree with Exablaze's general thrust that my chances of success are dim but perhaps not for the same reasons. I certainly believe and hope that my view of justice will be agreed to by the Appellate Court but my personal history suggests that my likely outcome at the FCA will be yet another defeat. Lay self represented litigants are at a considerable disadvantage and have very little track record of success. Even if legal aid was available, I can't imagine a pro-bono lawyer getting too worked up about a fight between two HFT types regardless of the injustice involved. Lawyers serve a good purpose but they have to eat and can not represent poor people like myself. I am left to do what I can with the support of family and friends.

Some statements in the Zomojo / Exablaze communique are dubious at best. Let me meander though my thinking as to why.

One option of remedy being put to the Appellate Court is a constitutional writ of Certiorari. If this was granted by the court the entire case, including the Zeptonics bits, could be tipped out. I hope this happens and believe that would be just, but, yes, the historical record of my legal defeats suggest this is unlikely. The court is not a justice system. It is a contract resolution system where $ assist greatly in providing supportive argument.

This writ of Certiorari is different to the US use of the term where is it seems to be an order from an appellate court requiring the case to be forwarded. In Australia such a prerogative, or constitutional, writ is a remedy, quite a heavy handed one, that may result in the decision being overturned and details struck from the record:
"Certiorari is an order setting aside a decision (technically, the record of the decision-maker is removed to the court and the court then quashes the decision and expunges it from the record). An order for certiorari would be sought where a decision has been made unlawfully and the decision should be set aside." [Source: The Law Handbook web site.] 
Zomojo argue in their communique that Zeptonics is not represented thus decisions that have affected Zeptonics cannot be altered. This is not true as such a writ could result in the case being wholly undone. Also it is important to note that virtually any party, even a party unrelated to the original case, may attempt to appeal a court's decision by seeking such a writ if meritorious. For example, if the London whale had of been found guilty in the FCA in Melbourne, Greenpeace may have been entitled to run an appeal based around such a writ as an unrelated and very confused party.

More technically, the standing of a person seeking such a writ should be that of an “aggrieved person” which, according to the Queensland decision of R v Knyvell; ex parte Weber ([1929] St.R.Qd. 16; and see De Haber v. Queen of Portugal [1851] 17 Q.B. 171) could be as wide as “any person” following the tradition that unlawful dealings of tribunals under the cloak of law should be allowed to be challenged by any subject. The history of this relatively loose requirement for standing culminated in the decision in Attorney-General of Gambia v N’Jie ([1961] A.C. 617.), where the interpretation of the term “person aggrieved” was effectively “any person.”

Maybe my legal thoughts are off base and I'm barking up the wrong tree. Perhaps, just maybe, there is some legal merit to this line of reasoning. The court will adjudicate in due course if I can cross the t's and dot the i's and make it that far.

Zomojo argued vigorously to Honourable Justice Middleton that the appeal not be allowed to continue. The process was allowed to continue. Their QC did not win that argument, at least yet. The Full Bench of the Federal Court of Australia will hear the leave to appeal and appeal on a single day in 2015, likely to be February or March.

If you're having trouble sleeping, you can read my draft notice of appeal and a further submission from myself as they are public documents registered with the court.

I believe Zomojo is being a little misleading in their communique. They didn't really need to be as I acknowledge that, regardless of the merit, a self represented litigant is unlikely to succeed. I fervently hope to be an exception to that general rule as the scores of people harmed by the injustice of this mess deserve better.

--Matt.





Friday, 19 September 2014

Zomojo v Zeptonics appeal going to the full bench of the FCA

I've not had much success in the Federal Court of Australia but a small ray of hope broke through the gloomy clouds yesterday. A very small ray. Honourable Justice Middleton permitted [though maybe didn't prevent is a better synopsis] my appeal to move forward to the Full Bench of the Federal Court of Australia with a likely schedule of February or March 2015.

No, it's not the appeal I wished I could present, but at least the Court has had the good grace to allow this piece of due process. Even though it feels a bit like a lamb to the slaughter, perhaps, just perhaps, the Judicial Officers of the appellate court will be able see the underlying truth and find a way to rebalance the scales the justice.

Winning the appeal would indeed be a Pyrrhic victory as the harm that has come to pass cannot be undone. With the good ship Zeptonics destroyed, scores of people and their families affected, there can be no proper accounting to undo the evil. I do hope Zeptonics can get the set of its intellectual property back that was ordered by the Court to be assigned to Zomojo, some of which is being used by Exablaze, and that dependent customers are not overly affected as they were in the sinking of Zeptonics. Obviously I believe I have a good case and hope for the best, but my personal court history suggests my chances are slim.

Being broke and trying my best to be an orderly self represented litigant, I have much to do over the next little while to fight a proper appellate battle. I do suspect I'll have a busy Christmas and an unhappy wife. If there is any one out there with suggestions or formal legal advice, apart from “Are you completely mad?”, please drop me a line on matthurd@acm.org
“Never give in. Never give in. Never, never, never, never—in nothing, great or small, large or petty—never give in, except to convictions of honour and good sense. Never yield to force. Never yield to the apparently overwhelming might of the enemy.” 
― Winston Churchill, Never Give In!: The Best of Winston Churchill's Speeches

--Matt.

Wednesday, 6 August 2014

Historic US microwave links and NY to Chicago in "34 Jumps"

Recently Jump Trading was in the news with their NATO tower story. It seemed to be well read with quite a few different brokers around the world mentioning it to me.

However, not many people really realise how old the news is. If you had of searched for a story with the words "Jump", "microwave", New York and Chicago, you may have come across this link as one of the first:

Click to expand. Source[1]: Originally September 1949 Long Lines Magazine
Yes, that's right. September 1949. A whole sixty five years ago next month. The second page of the article is below at [2].

A New York to Boston link went up in 1948 [3]. East joined West with the "Skyways" intercontinental link, with obligatory White House telephone call christening on August 17, 1951. It referred to operations at four billion vibrations a second[4], so presumably it was 4GHz microwave technology.

The network grew in size and here is the footprint of the US National "Broadband" network in 1960:

Click to expand. Source[5]: US 1960 Broadband Network
Microwave travels at around 3.3 microseconds per kilometre and light through optical fibre at around 5 microseconds per kilometre. AT&T's first "slower" long line fibre link went live in 1983 / 1984:

Click to enlarge: AT&T's first long fibre link
We have to be careful about books like "Flash Boys" that twist history into falsehoods to make catchy plot points. I guess we'll be hearing about new TransAtlantic high frequency trading stories soon, but remember that the first cable went across in 1858. There is a long, rich history of incredible stories in telecommunications that more people should be aware of.


Click to enlarge: Source [7]

I'll let you into a little known secret: that early transatlantic cable may have been 3 to 4 milliseconds faster than the current fastest fibres for signals crossing the ocean. At least for the three weeks of operations it had. Ask an engineer why that may have been the case...

Happy trading,

--Matt.

_______________

[1] Sourced from a cool site about the old long lines: http://long-lines.net/

[2] The follow up page from [1]

Click to enlarge. Source www.long-lines.net
[3] From long-lines.net the Boston NYmicrowave link article from 1948:



[4] Nice pamphlet on the 1951 Skyways originally from long-lines.net

[5] Original 1960s map from long-lines: http://long-lines.net/places-routes/maps/MW6003.html

[6] An earlier long-lines map via long lines from the Bell System Technical Journal, September 1952:

[7] Early Transatlantic Cable from from http://atlantic-cable.com/Maps/index.htm

[8] Here are a couple more maps from [7].



Tuesday, 15 July 2014

Mellanox UDP latency = 1.1 microsecond

Mellanox's VMA release notes for VMA version 6.6.4 have updated Netperf UDP RR benchmarks. The table below is  for one of their Ethernet cards.  1.1 microsecond is a good result for a well priced and well supported card with a comprehensive feature set.

Not quite Dolphin PCIe territory where they claim 0.74 microseconds but Dolphin's socket implementation on top of PCIe (not Ethernet), if you chose to use it, would make the solution a little worse than Mellanox according to their Dolphin's supersocket reporting (1.26 microseconds).

Has anyone some public SolarFlare or Chelsio numbers? I'd love to compare.

Happy trading,

--Matt.




Monday, 14 July 2014

Zomojo / Exablaze developments - firing staff

Some interesting, to me at least, news about Zomojo / Exablaze the last couple of weeks.

I hear a bunch of staff have just been fired and are interviewing around town. In fact, I've been told by more than two sources now that Phil Manuel, the SA, is the only full time staff member left at Zomojo with some of the older crew working two days a week with the rest terminated. One of the guys I hired back in 2009 has a one month old baby and a two year old and is now looking for work. Harsh. He is a good guy. Zomojo is evil but there are some good, perhaps deceived, people involved I have considerable sympathy for. Good-luck to them and their families.

I guess it is not surprising to hear how Exablaze are now having yield problems with manufacture as quality problems might be more commonplace with a device you didn't conceive of that requires modifications (the management board was end of lifed, for example). Quality control, tsk, tsk. I can't imagine you'd be feeling comfortable if you were an Exablaze customer.

The two day a week part-time slavery for some staff is interesting. I wonder if it so they can avoid the long service leave entitlements most of them are due? Perhaps it is just a hook to entrap them and keep them from plying their craft elsewhere?

I understand Zomojo capitulated / settled with a trading firm they were suing last year in the first half of 2013. As far as I can tell from the court files, that firm, a completely innocent party, still hasn't been paid their due costs of some hundreds of thousands of dollars for their loss as agreed. That has been drifting out for over a year now. There is another court question this Thursday regarding paying for an IT firm Zomojo had work done last year as part of their legal case against me and that firm has not been paid for over a year now for work done. PPB Advisory should have been aware that if they were going to lie with dogs they may get fleas.

Makes you wonder if Zomojo is solvent?

Then, just now, this article turns up in Korea: FUTURES AND ZOMOJO?! (A better English translation is now below at [2]) I think they may have the wrong broker but the gist of the story may be true enough. The broker mentioned is a solid broker and it would be a shame if they are suffering from an unwarranted Zomojo contagion.

Zomojo lie in court. Zomojo mislead their customers. Greg Robinson was terminated by ITG. Robinson's Bellwether Share Fund investment and directorship didn't turn out too well. Now he has driven Zomojo off the side of the road. He is indeed a man with a black thumb for the corporate garden. If you hear his footsteps behind, you may be justified in quickening your pace.

I do feel a bit sorry for Robinson as one day his family will wake up and realise he is just a pseudo-intellectual fraud.  He is good to have a beer with, pleasant to talk to, have a round of golf with, but deeply flawed nonetheless. A person of no substance. Taking my wife's super from a decades of teaching. Vilifying my father unjustly after went to great lengths to assist them. Robinson is trying to bankrupt the good citizen in his twilight years. Causing my family enough hardship and stress to put one of my daughters in hospital. Nasty. I guess I'm a little biased, so make up your own mind. Caveat emptor.

The second director, Dean Boyle, was just a lap dog sniffing Robinson's butt. However, it is Ian Heddle who is the true despot in that place.  It is not really worth any energy or thought wasted as whereas Robinson is more akin to a knife carrying sneaky fart, Heddle is just a plain sh.... Obvious on sight.

I still hear ex-Zomojo people refer to Heddle as Bubble. I thought the nickname was just because of his bulbous head which is out of proportion with his already large frame. Being above the fray (can't you tell) I never questioned the origin of the nickname. After about a year, I found out they called him that because he was like the boy in the bubble (with apologies to David a heroic lad). Heddle came in to work, never did anything, never interacted and left before anyone else. He lived in his own little bubble. He'd vilify staff and not talk to people for weeks on end, including myself. This is why I left. It was a soundly unpleasant place to work and it got to the point where you just didn't want to turn up to work. Leaving was both the hardest and the easiest decision.

I think the funniest story I've heard about that horrid time is the game some of the staff used to play, "Guess what Bubble is doing today." Each day they'd have a sweep on what he would be doing at exactly 11am, as he didn't do a great deal at all. It got a bit monotonous as each day it turned out to be browsing the web. Every day. All day even. Not just 11am. Worse. The guys couldn't break it down much further as it turned out to be almost always cartoons. Not the graphic novel, mature, artistic kind. The simple kiddie stuff in the main. The game was retired as it turned out to be a closed ended problem and of no scientific merit or wagerable outcome.

I can't imagine why Zomojo no longer makes money and have reduced to a skeleton staff . Can you?

Happy trading,


--Matt.



_______________________

[1]
PS:  I still own a third of Zomojo Trading Pty Ltd. No annual reports. No information at all. It is the vehicle through which all the revenue comes in, or at least it used to be that way. They seem to be violating a bunch of ASIC and ATO rules. Not sure what to do with the holding as I certainly can't afford legal advice any longer.  If you have any creative ideas, let me know ;-)

[2]
“Hyundai futures providing unfair advantages via FEP?” HF states ”this is a false claim” Claims of Hyundai Future providing its foreign clients with methods of unfair advantages of FEP and lease line, giving them an advantage and harming normal investors, have been circling the market. But Hyundai Future has stated that this was a false claim.

“A” securities firm director “B” stated, “Hyundai Futures has provided a FEP with direct connection to KRX’s order system to Zomojo for a monthly fee of KRW 200 million”. The director also said “Zomojo directly manages the servers and even install their own programs and place unfair trades”

Zomojo is known to be an algorithm trading firm using their programs. FSS regulation states that firms including Hyundai Future are prohibited to provide clients like Zomojo with special services, information or equipment.

It is also prohibited for the clients to install their own programs as Director “B” has stated. This gives the special clients the advantage by having higher speeds than KRX’s standard compared to normal investors.

On the other hand, Hyundai Futures stated that they have never provided Zomojo with the servers or even opened an account with them. HF representative said, “we are aware that Zomojo is a specialist in the FEP field, and we have met with them, but it was because Zomojo proposed to sell their software to us.”

“We have declined Zomojo’s offer, and the claims that HF is using Zomojo’s software is false.” It has been identified that Zomojo has reached out to other firms as well as HF. Therefore, it is a possibility that other firms also provide services in an unfair manner such as HF.

Person in the brokerage industry stated, ”it is tough to identify if the securities or futures firm is utilizing these programs in the servers without visiting the sites.” “foreign algorithm trading accounts in Korea has reached over 1200 accounts since March”

A KRX personnel said, “only way to identify these illegal algorithm trading is to inspect their IDC directly”, “but even if we charge the firms with suspicion, the firm will simply decline to provide information stating it is their confidential business information.”

FSS personnel said “we have sent guidelines as this has been a problem before”, “instead of criminal charges, it would be easier for the government to address disciplinary actions through administrative measures”

Tae Woong Ryu Reporter (Aju News)

Sunday, 15 June 2014

Nice Terasic PCIe 3.0 FPGA board

I do like the look of this board, the Terasic TR5-Lite even if it is not quite as sexy as a Triumph TR5 from my birth year. Any feedback on the actual use of one of these would be most welcome.

Stratix V, half height, half length with a firewire (1394) shaped 422 connector for timing input. Direct connectivity to the dual SPF+s. 36MB QDRII+, 2GB DRAM and PCIe 3.0 capable from the x8 edge connector makes it look good for HFT.

It uses a Altera Stratix® V GX FPGA (5SGXEA7N2F45C2) so you get 622k logic elements. Dunno what it is, but the board just looks nice and elegant. It only seems to have 4 user controllable LEDs  and five status LEDs, so Mike Dini wouldn't approve[1].

Whilst an Arria V or Kintex-7 solution would be better value for 10G, Terasic seems to have put together almost the perfect minimal FPGA HFT package here. I'm still thinking a Zynq-7100 based board with its useful ARM is a better idea overall though.

--Matt.



Friday, 6 June 2014

Finra ATS stats

Finra has made a stab at releasing ATS / dark pool stats.

It makes for some interesting reading. I've put the stats into a table below as I've found the Finra web site has been a bit up and down. Is that ITG and IEX running neck and neck with 100M shares a piece?
Finra ATS / Dark Pool Stats - Source: Finra
(Note the % are bunk, though indicative, as the dates are different)

Name Shares
Trades
Shares/Trade Date
CROS CREDIT SUISSE SECURITIES (USA) LLC 376,207,777 12.1% 2,189,977 13.5% 171.8 06/05/2014
LATS BARCLAYS CAPITAL INC. 307,354,920 9.9% 1,701,372 10.5% 180.7 06/05/2014
UBSA UBS SECURITIES LLC 282,217,511 9.0% 2,307,746 14.2% 122.3 06/05/2014
MLIX MERRILL LYNCH, PIERCE, FENNER & SMITH INCORPORATED 215,734,880 6.9% 1,099,830 6.8% 196.2 06/05/2014
MSPL MORGAN STANLEY & CO. LLC 207,196,400 6.6% 780,930 4.8% 265.3 06/05/2014
DBAX DEUTSCHE BANK SECURITIES INC. 189,134,200 6.1% 1,117,870 6.9% 169.2 06/05/2014
SGMA GOLDMAN SACHS EXECUTION & CLEARING, L.P. 182,887,381 5.9% 1,094,422 6.7% 167.1 06/05/2014
FLOW LAVAFLOW, INC. 154,595,446 5.0% 862,489 5.3% 179.2 06/05/2014
KCGM KCG AMERICAS LLC 135,074,012 4.3% 979,475 6.0% 137.9 06/05/2014
JPMX J.P. MORGAN SECURITIES LLC 113,052,036 3.6% 422,060 2.6% 267.9 06/05/2014
ITGP ITG INC. 106,841,795 3.4% 415,319 2.6% 257.3 06/05/2014
IEXG IEX SERVICES LLC 101,722,565 3.3% 304,784 1.9% 333.8 06/05/2014
BIDS BIDS TRADING L.P. 96,849,500 3.1% 221,827 1.4% 436.6 06/05/2014
KCGB KCG AMERICAS LLC 65,498,506 2.1% 574,660 3.5% 114.0 06/05/2014
ICBX INSTINET, LLC 64,843,400 2.1% 269,894 1.7% 240.3 06/05/2014
EBXL LEVEL ATS 60,250,900 1.9% 378,885 2.3% 159.0 06/05/2014
CXCX CITIGROUP GLOBAL MARKETS INC. 57,960,326 1.9% 300,479 1.9% 192.9 06/05/2014
DLTA DEALERWEB INC. 55,129,108 1.8% 103 0.0% 535,234.1 06/02/2014
XSTM NATIONAL FINANCIAL SERVICES LLC 42,356,635 1.4% 113,495 0.7% 373.2 06/05/2014
LTPL CREDIT SUISSE SECURITIES (USA) LLC 34,167,610 1.1% 287,279 1.8% 118.9 06/05/2014
IATS INTERACTIVE BROKERS LLC 33,835,637 1.1% 97,090 0.6% 348.5 06/05/2014
LQNT LIQUIDNET, INC. 33,593,400 1.1% 811 0.0% 41,422.2 06/02/2014
APOG CITADEL SECURITIES LLC 30,638,504 1.0% 180,977 1.1% 169.3 06/05/2014
NYFX CONVERGEX EXECUTION SOLUTIONS LLC 26,367,524 0.8% 60,292 0.4% 437.3 06/05/2014
MSTX MORGAN STANLEY & CO. LLC 23,245,500 0.7% 141,446 0.9% 164.3 06/05/2014
PULX STATE STREET GLOBAL MARKETS, LLC 19,573,064 0.6% 1,743 0.0% 11,229.5 06/02/2014
VRTX CONVERGEX EXECUTION SOLUTIONS LLC 14,949,562 0.5% 69,232 0.4% 215.9 06/05/2014
PDQM PDQ ATS, INC. 14,866,289 0.5% 79,138 0.5% 187.9 06/05/2014
LQFI CITIGROUP GLOBAL MARKETS INC. 13,779,198 0.4% 242 0.0% 56,938.8 06/02/2014
INCA INSTINET, LLC 13,059,700 0.4% 16,985 0.1% 768.9 06/05/2014
WELX WELLS FARGO SECURITIES, LLC 11,996,652 0.4% 68,802 0.4% 174.4 06/05/2014
LQNA LIQUIDNET, INC. 9,010,800 0.3% 3,586 0.0% 2,512.8 06/02/2014
BOOK BLOOMBERG TRADEBOOK LLC 8,655,811 0.3% 39,562 0.2% 218.8 06/05/2014
MLVX MERRILL LYNCH, PIERCE, FENNER & SMITH INCORPORATED 5,508,700 0.2% 13,224 0.1% 416.6 06/05/2014
AQUA AQUA SECURITIES L.P. 5,072,599 0.2% 305 0.0% 16,631.5 06/02/2014
LEHM BARCLAYS CAPITAL INC. 2,913,308 0.1% 20 0.0% 145,665.4 06/03/2014
RCSL RIVER CROSS SECURITES, LLLP 1,330,693 0.0% 7,266 0.0% 183.1 06/05/2014
MSRP MORGAN STANLEY & CO. LLC 1,115,700 0.0% 2,112 0.0% 528.3 06/02/2014
FNBR FOLIOFN INVESTMENTS, INC. 379,218 0.0% 11,191 0.1% 33.9 06/05/2014
WDNX WEEDEN & CO.L.P. 308,590 0.0% 514 0.0% 600.4 06/02/2014
CURX CURIAN CLEARING, LLC 184,094 0.0% 3,787 0.0% 48.6 06/02/2014
JEFX JEFFERIES EXECUTION SERVICES, INC. 6,700 0.0% 56 0.0% 119.6 06/02/2014







Total 3,119,466,151 100.0% 16,221,277 100.0% 192.3



Friday, 2 May 2014

100G Ethernet FPGA NIC - PCIe 3 with 16 lanes + GPS

I hadn't heard of these guys: Inveatech. Shame on me as the spec's look neat. Though I'm not quite sure why they're reselling the Stanford NetFPGA-10G cards. Those NetFPGA-10G cards are not my favourites due to their power and clock specifics as well as their PHY latencies. But I digress, here is a picture of their cool 100G NIC:

Inveatech 100G card: - 1 x 100G or 10 x 10G
They are not publicising how much QDR and DDR3 the 100G NIC carries. A Xilinx blog tells us which FPGA they're using, 
"the INVEA-TECH COMBO 100G HANIC accepts one 100Gbps CFP2 optical Ethernet transceiver module and the on-board Virtex-7 H580T 3D FPGA receives the Ethernet streams using four of its GTZ 28.05Gbps SerDes transceivers operating at 25Gbps to communicate with CFP2 cage"
That's an expensive FPGA, so the cards will not be cheap.  Their 80G card with 2 x 40G QSFPs uses a Virtex-7 H690T for what it's worth. Interestingly, the 100G card must have an active module to pull out 10 x 10G lanes from the 4 x 25G lanes if the vendor and Xilinx descriptions are both right.

They also claim to have some interesting trade acceleration IP bits and pieces. I'd be interested in hearing from anyone who has had any experience of these tasty looking Czech morsels?

If you're thinking of just playing then you're probably better off getting one of the Xilinx dev boards. It is hard to go past their $5k 4 x 10G Virtex-7 board which includes node locked tools.  

The distributor High-tech Gobal has a couple of neat morsels for the price or space constrained:
100G will bring more heartache. *Sigh*, so many 100G module form factors to choose from. There are six main ones. "Standards" are always just dandy when they provide so much choice. I'm just hoping that when we do 100G we can stick to QSFP28 modules instead of CPAK, CXP, CFP2 or CFP4 modules, though the density of the HD modules are tempting even if we'll have to buy new cabling.

--Matt.

Tuesday, 8 April 2014

Flash Boys' sinister ending: who owns the new evil microwave tower?

At the conclusion of Flash Boys, Lewis holds out a challenge to find who owns the tower with the mysterious FCC license plate number 1215095.

"The application to use the tower to send a microwave signal had been filed in July 2012, and it had been filed by ... well, it isn't possible to keep any of this secret anymore. A day's journey in cyberspace would lead anyone who wished to know it into another incredible but true Wall Street story, of hypocrisy and secrecy and the endless quest by human beings to gain a certain edge in an uncertain world." 
 Part of the last paragraph of Flash Boys

This final piece of the narrative paints this particular microwave New New Thing as the next, somewhat sinister, step in the evolution of speed. Spread Networks was one piece of secretive infrastructure used to help rig the markets. Now it is the turn of evil microwave. Who owns this new form of evil to help the HFT pirates continue to rig the financial markets?

The internet has reacted! A bit of frenzied internet sleuthing has subsequently taken place over a number of blogs, tweets and web sites. A number of parties have reached the same conclusion.

Tradeworx! Mr Narang confirmed it to the WSJ. They're the new bandits!

Sigh... I don't think so. Read on.

I found the ending of Flash Boys one of the saddest bits of the Lewis book. I feel it was sad for journalism, unless you support deceptive journalism.

Mr Lewis goes to great pains to weave a narrative around Spread Networks. I think you'd have to agree it is a key part of the plot. He then throws out the New New Thing of microwave at the conclusion of the book with clear sinister overtones. You could almost hears the Jaws theme in the background. Good people have been hunting down who could it be doing this New New Thing of evil that was the evolution of speed after Spread Networks? A participatory cliff hanger to ferret out the perpetrators was a neat literary device.

Surely it would have been responsible journalism for Lewis, especially when writing on the scale of a book, a Lewis book nonetheless, to “google” the topic so he could have found out that his narrative was wrong and RF microwaves had been used on the Illinois – New Jersey link since 2009, BEFORE spread networks went live.

It took me less than thirty seconds to get a reference to a credible microwave 2009 story. Here's an article I've tweeted previously from the Chicago Tribune from 2012.
"He [Benti] said the microwave network starts at 350 E. Cermak, ends at another telecom hotel at 165 Halsey St. in Newark, N.J., and went live in the fourth quarter of 2009."
Did he ask Spread Networks? Lewis can remember hand movements and idle moments from conversations of meetings with enough clarity to include much minutiae in the book. Maybe Spread Networks didn't know about microwave when Lewis talked to them?

From page two of the Chicago Tribune article,
"Spread Networks CEO David Barksdale dismissed such criticism. Yes, data travels faster through the air than through fiber-optic lines, but severe weather events can disrupt microwave signals, and those signals can't carry as much data at one time as fiber can."
I think they knew.

Is it shoddy journalism from Mr Lewis? Perhaps he didn't want these particular facts to get in the way of his significant plot device. Now, if Lewis had included the microwave links from 2009 in the story, it wouldn't have necessarily changed his message. He should have been more aware and not so inaccurate. If Lewis can't get the basics of his main Flash Boys narrative right, you need to take everything else with a pinch of salt.

Also, why is Tradeworx copping flak for this? This fact of Tradeworx owning microwave was been public for sometime. Wired ran a high profile story in 2012 that even had a big coloured box with the Tradeworx name on it. There was even a price tag! Hardly a complete secret.

From the wired article with Tradeworx hidden in plain sight
In the words of the great Lleyton Hewitt, “Come on!”

There is no news here, just scuttlebutt and harmful innuendo. Lewis is implying Tradeworx is rigging the market. They are not. They're just collateral damage in Lewis' missteps.

How can we trust the story telling if the main plot device is so twisted out of shape?

Ought Lewis have known more about microwave?

Jim Barksdale is the investment force behind Spread Networks. Is he also an investor in IEX? Can't be sure, but his fellow ex-Netscape CEO Jim Clark is on the IEX board. Obviously these guys are acquaintances of Lewis from the New New Thing days. A Barksdale, David, remains CEO of Spread. Lewis could have asked them. Any of them. Based on the Chicago Tribune story, they ought to have known.

Lewis didn't even need his contacts here. He could have clicked "search."

Incompetent or deceptive?

You decide.

--Matt.




_______
An example of the disclosed use of Spread Networks fibre:

KNIGHT HOLDCO, INC. S-4

Getco use of Spread from their S-4:
 
"Colocation and data line expenses increased $18.9 million (52.0%) to $55.2 million in 2010 from $36.3 million in 2009 primarily due to the introduction of Spread Networks, which is a fiber optic line that transmits exchange and market data between Chicago and New York, and the build out of GETCO’s Asia-Pacific colocations and data lines."

Monday, 7 April 2014

HFT - two choices: making money every day or oblivion

If an HFT makes money every day the system must be rigged!

It's not rigged.  It's just math!

I see since Flash Boys came out, HFTs have been copping flak for making money consistently.

Virtu had one down day in 1200 or so days. It must be rigged.

Well, the simple fact is that if you don't make money every day, then you're doing it wrong!

You do, or you die. It's that simple.

HFT implies that you do a lot of trades. If you do a lot of trades with a smidge of favourability, you should always be winning. It's just simple mathematics. The law of big numbers. A systematic screw-up for a day may cause an odd loss but that is a different risk category. You seriously wouldn't be doing a zillion trades of anything without an overall positive expectation. The law of big numbers also works against you. If you're on the wrong side of the fragile line you're practically guaranteed to lose.

This is not to say that you have a "cooked" system where every trade is a winner.

Let's look at some numbers:

Say you have a trade that makes you $0.01 per trade when you get it right and you lose $0.011 per trade when you get it wrong.

Let's now assume you do 10,000 trades a day.

What winning trade % do you need to win at for almost every day to be a winner?

Let's look:
  • 51% means roughly 99.7% of days are losers, a bit over 0.2% of days are winners
You probably wouldn't be happy with that.
  • 53% means 89% of days are winners
  • 54% means 99.9% of days are winners - you lose one day in four years
  • 55% means you really shouldn't lose
The percentages are tight. It is a matter of getting the profit / loss distribution for your trade outcomes right so a lot of trades make it hard to lose money.

Fundamentally, if you're a high frequency trader, you're doing a lot of trades. That's pretty much the definition of high frequency, otherwise you're a low frequency trader. A lot of trades are silly if you don't have a positive expectation. If you have a positive expectation then a lot of trades means you really should have a very high chance of winning on every day even if each trade's win chance is a bit like the toss of a coin.

Let's look at the same situation if you only did 100 trades a day. A hundred is quite a few but not really high frequency.
  • 51% trade win probably = about 38% of days are winners
  • 54% = about 62% of days are winners
  • 55% = about 69% of days are winners
Not quite as good as having a lot of trades at 55%. However at 51% you were previously pretty much guaranteed to lose with lots of trades but you may live a few days longer with only 100 trades per day. This is also why if you want to bet on red or black at roulette, where the house has an edge thanks to zero, you should just make one large single bet to maximise your chances of not losing.

You can see that lots of trades help improve the guarantees. You win or lose more consistently.

The hard thing about HFT is that if it was easy, everyone would do it. There are no easy trades with the level of sophistication that exists in mature markets. It is challenging and you are constantly being pushed to the margins as someone else is always prepared to make a little less and take all your edge if they can. HFT market makers fight compulsively over the scraps and win by making less than another team. It's a bare knuckle fight against other market makers where your reward is to make less money than your opponent if you win. You in turn get beaten by someone prepared to make even less. You are scared of large trades or better informed traders that will trade through you and screw with your distribution of returns. There is little room for error.

It is simple. An HFT should be making money every day or they will be going out of business. There is money or there is oblivion. There is never any forgiveness, just paranoia.  HFTs are terrified of adverse selection. Worse still, other market makers are trying desperately to replace you by being a fraction better. Lots of smart firms are always shutting down in HFT land. It is a tough business.

So, what's up with traders making money every day?

It's not rigged.  It's just math.

--Matt.



PS: Survivorship bias for HFT types means that all should have consistent profits if they remain in business for an extended period of time. I find that an interesting thought.
___________

Earlier description of how HFT market makers make money
Earlier reasoning as to why investors and speculators can both win in markets

___________

Footnote:  I spent quite a few years as a positioning style trader with a global portfolio of currency, short and long interest rates, equity indices and some commodities.  I made money every year but it was a battle.  Each day was a 48% chance of a win. So, overall I was usually wrong. I was successful because the average winning day was 1.6 times the average losing day. It was pretty scary though as just missing your dozen best days in a year would be the difference between an annual profit or an annual loss. It was also quite funny when people asked for comment on the markets as it was hard to explain that with my trading I was more likely to be wrong that right for any given day.

HFT is pretty similar. It is also just a game of profit and loss distribution. It is a lot easier on the psychology as instead of waiting for a whole year to see if things work out, you should know at the end of each day with a large enough number of trades. Even worse for the psychology is that as a position style trader normally the profits are positively autocorrelated so that if you are losing you expect to continue losing. Worse, you can't do anything about because if you over trade, resulting in extra frictional costs, you're only guaranteeing an eventual loss. HFT has better daily psychology but higher paranoia as there is a lot that can go wrong.

___________

Footnote: This is vastly simplified but correct in essence.  Profits and loss tend to be positively autocorrelated which makes things more volatile in practice. Distributions are nowhere near as simple which is why there are PhDs running around with computers attached to their fingers.