After more than 4 months of researching and DD.
We need a serious change.
The DTCC needs to disappear: No more private owned (by the elite and 1%) self regulated institution full of greed and corruption taking advantage of businesses and retail.
No more IOU’s: No more virtual assets or imaginary shares. Back to physical or completely transparent system, if you buy something you own something and can claim it anytime, no more middleman holding assets forever.
SEC needs a reform: Real reform and real power fully transparent and fair. No more going on favor of banks approving rules favoring them and giving exceptions to favor them. They need to be fair and partial to keep a healthy market and economy.
Blockchain system: a completely transparent, honest and fair system that favors all parties in a win-win-win scenario, honest with no massive gambling bets and simple. If the company grows then the owners or the company benefit, if the company is in trouble and is going bad so is the stock.
Wealth distribution: The money the DTCC (elite and big money) has been stealing and taking from businesses and retail for years needs to come back to the people they disrupt for so long, all the jobs, houses, struggles, technology lost and advancements everyone lost because of that elite greed and control needs to be paid.
It’s time, we need to wake up and make this world a better place. Enough is enough.
No financial advice, this is what needs to be done so we can recover as society and have a better world. Starting from the economy.
Edit: The saddest things about this whole situation is, we are being fucked by the middle man. This middle man are banks and big money that control and bought the government.
They are taking advantage of the transactions when all we want to do is support businesses and markets and at the same time benefiting for that support and help.
Edit: video worth to watch https://www.youtube.com/watch?t=10&v=Kpyhnmd-ZbU&feature=youtu.be
They never learn…
As the title states, I'm a distributed systems engineer working at a FAANG company. I recently researched Palantir. This is my analysis of the company.
TLDR: it's a buy
Before we explore Palantir's potential, we first need to understand what it does. Its product offerings are complex and highly technical, and I've noticed that many analysis articles gloss over how they work. In doing so, they greatly miss Palantir's story and either exaggerate or underestimate its potential.
A 30,000 Foot View
Palantir is a unique data-focused Software-as-a-Service (SaaS) company. Competitors include Snowflake and Tableau. However, Palantir's products / customers are well differentiated from the competition. Palantir just thinks differently and it's paying off in spades.
At a high level, Palantir has a commercial and government business. The flagship product for its commercial business is Foundry, while the flagship product for its government business is Gotham. Although the government business is solid with little competition, its commercial business is where everyone expects growth to come from.
Commercial Business (Foundry)
Foundry is a feature-rich end-to-end data analysis system for companies. It tries to (fully integrate) with a company's data, in order to provide much richer insights.
Full integration is how Foundry differentiates itself from the competition. The competition likes to build generalized software that accomodates disparate schemas but can only perform shallow analysis (e.g. Snowflake for scalable data storage/processing and Tableau for data visualization).
Foundry, on the other hand, takes a company's data, and transforms it into an internal ontology (data language) that it understands. Once the data is transformed, Foundry deeply understands the data and can provide much richer insights than the generalized software of the competition.
The main problem with Foundry is that there's a lot of friction to integrate with a "deep integration" product. A company's data is usually all over the place, in diff databases with diff schema and its often inconsistent or incorrect. Foundry needs to ingest all this disparate data sources, then correct any problems, before transforming the messy data into its ontology. That's expensive.
However, its 2021 Q3 earnings report shows that Foundry can scale... the customer base is growing quickly while costs are going down. Quarterly commercial revenue grew by 37% year-over-year, with US quarterly commercial
... keep reading on reddit ➡Hello to all!
I am currently developing a web application being a distributed operating system and I would like to have your feedback on the subject.
The idea is that today, we have multiple ways to represent a product or a service on a web page. The techniques and concepts evolving rapidly in the front-end domain, I had the idea to start again on the basis regarding the operability of a system. So I developed a web site taking the form of a command terminal.
Most of the Linux commands are implemented, so you can create folders and files in your directories as soon as you are connected. In the future, I want to provide video, image and audio playback applications, as well as make a torrent client.
The advantage of being on a distributed OS is that the filesystem of all users can be shared, which means that you can visit your neighbor's directory (except for the hidden folders).
Here you go, I hope you enjoy and I'm waiting for your feedback :)
https://os.directory
I'm interested in starting a new project, but have been having some trouble coming up with inspiration. I've professionally worked on distributed systems in C++ at a couple jobs now (although not from scratch), and would love to use this both as an opportunity to learn more of the fundamentals and to create something at least somewhat useful.
So you have a bunch of things (microservices, databases, upstream and downstream dependencies).
When something is broken you want to know what broke and why and how to fix it. To do that, you need to know what all the 'things' are, and how they interact with each other.
The most generic "solution" to this problem that I've seen is Distributed Tracing. But that doesn't actually seem to solve the problem. With DT, you can look at a live view of a failing thing and try to trace the transactions along the way. But that doesn't tell you what things outside of that transaction are still involved and may have impacted the failure. If your API selects a row in a database table, but some other service modified that row earlier... the other service modifying that row isn't going to show up in your trace, is it? (yeah, that's not real microservices, but a similar idea happens when you're calling external services and those get modified by other external services)
A more straightforward approach (to me) would be to have something like an OpenAPI doc for every component of the system, and map out dependencies between all components, and generate documentation (and a DAG) that maps them all out. So you can look at the docs / walk the DAG and see all components that might be involved. But of course... OpenAPI doesn't seem to include that functionality (it has ways to list servers and links but it seems more like it's intended to reference what hosts your API is running on, not mapping independent services/dependencies together).
Am I missing something totally basic? How do other people map out their dependencies, both granularly and high-level?
Background - I work in the development team of an enterprise SQL database (that you might have heard of). YOE - 2. My work is mostly focused on some trivial incremental features and I think I am missing the big picture.
How do I learn more about the architecture of highly distributed and high fault-tolerant systems?
I have figured out a book - "Designing Data-Intensive Applications" [Kleppmann, Martin]
But what else can I do? Is only reading enough? How about practical exposure?
Hi All, I'm taking a classic graduate course from MIT https://pdos.csail.mit.edu/6.824/ on my own (I'm not enrolled, just doing the course independently). There are 4 different labs in Go lang I plan to redo in Scala (MapReduce, Raft, Fault-tolerant Key/Value Service, Sharded Key/Value Service). There is also a bunch of lectures available on YouTube: https://pdos.csail.mit.edu/6.824/schedule.html. I would like to know if there is any interest in forming a study group to motivate each other to complete this course and work on the labs together.
I'm looking to get into Quant Trading or Research and was wondering to what extent knowing how to implement distributed systems is important on the job? I know that this is mostly a SWE or Quant Dev's role, but I have the option of taking either that or Econometrics. I'm wondering which of the two would prove more useful to the job.
Hi, my background is Backend Engineer. But I have only experience is AWS SQS and SNS (for queue service).
My next company uses Apache Kafka with distributed system architecture. And I'd like to prepare to learn them.
By the way, I am looking for the Udemy courses, youtube, and articles. But there are a lot of them.
If anyone knows step by step level up course, articles or something like that, please recommend me. Thank you so much.
Subspace is a networking startup focused on routing internet traffic faster than the public internet. I've spent the last several years helping to architect and build what is now one of the fastest and largest IX inter-connected networks in the world.
My expertise and experience are primarily in distributed systems and network/routing technologies of all types, such as (but not limited to):
- BGP, eBPF. XDP routing
- TCP/UDP transport and security
- IPV4/6 routing and the differences between
- Internet Exchange peering and participation
- First, middle, and last mile packet transmission
- Solving complex business problems caused by packet loss and latency (particularly for large company networks)
AMA about the Internet! I'll answer technical questions from network developers/engineers, explain to people how to track down why their network is too slow, or pretty much anything else you ask me. Fire away!
Proof:
Twitter account: https://twitter.com/quentusrex
Blog post on my company site: https://subspace.com/resources/reddit-ama-session-internet
UPDATE: I would like to thank everyone for the questions and comments. The internet is an extremely complex and interesting topic, and I will be back to discuss this again in a few months.
In the meantime, you can check out our website at https://subspace.com or engage with us in /r/subspacepowered
Signing off for now, see you next time!
Hi,
I'm looking for free memory software like Ceph. However, it should support RDMA. At GlusterFS, support has unfortunately been discontinued. I have a lot of Infiniband hardware and don't want to use an IPoIB.
Can someone recommend a storage solution?
Hello,
I'm a junior backend dev trying to understand, at a high-level, the motivations and techniques associated with distributed database systems. I've made a few projects here and there, but I've never actually needed more than a single database for any of my applications.
Here are my questions:
-
In a distributed database system, do all of the databases just have a copy of the same exact data?
-
What is the limiting factor of a single database that would call for the need to horizontally scale? The maximum number of database connections per database?
-
It seems most shops are implementing 'event-driven' architecture. From what I've seen, it seems like when one of the database nodes receives a transaction, that transaction is typically logged in a NoSQL database and then broadcasted as an event by a message queue for the other databases to replicate. Does that sound about right?
Thanks a lot! I figured this was the most appropriate subreddit to ask since r/backendProgramming/ seems dead.
I am planning to apply to a masters degree in distributed systems and would appreciate any feedback with the SOP
Schematic of Power management and flight control system of AST satellites.
AST production satellites will feature the same central propulsion unit as the test satellite Bluewalker 3, this will provide AST with information concerning flight heritage.
Both these satellite models share the new form factor invented by AST & Science.
Power management and flight control has a high level of redundancy, as can be seen in the schematic above filed by AST to the regulating body, the FCC.
We see multiple solar arrays and multiple batteries, each monitored.
Then there are two power conditioning units, PCDUs, two low voltage controllers, LVCs, feeding 2 x 5 flight computers, FCs.
One of these 10 flight computers then can control: Two propulsion units, PPUs, (which incorporates Orbion Hall Effect - Ion -Thrusters), four reaction wheels, RWs, for attitude control and more than 1000 highly distributed array elements, M.
>These more than 1000 array elements all have their own "separate power, processing, sensing and actuating".
From interpreting this quote I find it likely that the antenna array elements called microns also contain an magnetorquer each, and that the "system on a chip" being developed for the Bluebird microns not only controls communications, but also "power, processing, sensing and actuating", these micron systems are entirely software defined on the Bluewalker 3 test satellite, but will be cheaper SoCs on the Bluebirds.
Microns are since before known to be sandwiched containing an earth facing antenna array for cellular fronthaul covered by the antenna elements needed for highly directive beamforming, an solar panel that is facing away from earth, if not both ways / bifacial, an heat insulating layer between these elements, system on a chip SoCs (which are Application Specific Integrated Circuits containing processors)
Separate power may mean separate power cables or something like distributed battery pouch cells in each micron and if so a distributed energy storage.
Apart from giving the spacecraft extreme redundancy and highly efficient attitude control for changing ballistic coefficient between high and low drag configuration without using propellant, this amount of magnetorquers all individually controlled opens up the possibility of some interesting applications, such as in flight array shape control i
... keep reading on reddit ➡A research team from Kwai Inc., Kuaishou Technology and ETH Zürich builds PERSIA, an efficient distributed training system that leverages a novel hybrid training algorithm to ensure both training efficiency and accuracy for extremely large deep learning recommender systems of up to 100 trillion parameters.
Here is a quick read: Kwai, Kuaishou & ETH Zürich Propose PERSIA, a Distributed Training System That Supports Deep Learning-Based Recommenders of up to 100 Trillion Parameters.
The code is available on the project’s GitHub. The paper PERSIA: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameters is on arXiv.
Limiting the sample to 2 different data nodes/ 2 services with their own databases, what are the common patterns for the subject problem?
The simplest of option of making writes to either of the services synchronously write to the other node is at the moment not feasible.
If it’s made asynchronous, write to service A might succeed and it’s propagation to service B asynchronously might fail. Or the other way around.
What patterns exist to solve this master-master replication problem? (I’m dealing with services using different flavors of databases)
This paper deals with how to implement a distributed control and monitoring system based on the Ethernet network in the microgrid. The paper describes a control strategy to implement both grid connected and islanded operation modes of the microgrid.
Welcome to Round 52 of the Cryptotrivial 2021 contest!
This is the last cryptotrivial until moons are distributed to not confuse people with governance pols.
Question 52: A particular method of asynchronous peer-to-peer communication between computer nodes of a distributed system.
The correct answer of Round 51 is...
Return on Investment (ROI)!
What is Return on Investment (ROI):
Return on Investment, or ROI for short, is a ratio or percentage value that reflects the profitability or efficiency of a certain trade or investment. It is a simple-to-use tool that can generate an absolute ratio (e.g., 0.35) or a value in percentage (e.g., 35%). As such, ROI can also be used when comparing different types of investments or multiple trading operations.
Specifically, ROI evaluates the return on an investment in relation to its purchasing cost. This means that the calculation of ROI is simply the return (net profit) divided by the total acquisition costs (net cost). The result may then be multiplied by 100 to get the percentage value.
Naturally, a high ROI value indicates that the investment was profitable, while a negative ROI means the return was lower than the costs. The calculation of ROI is based on the following equation:
ROI = (Current Value - Total Cost) / Total Cost
Alternatively, it may also be written as:
ROI = Net Profit / Net Cost
As an example, imagine that Alice bought 100 BNB for 1,000 US dollars - paying 10 dollars each. If the current price of BNB is 19 dollars, Alice would have an ROI of 0.90 or 90%.ROI is widely used in both traditional and cryptocurrency markets. However, it has some limitations. For instance, Alice may use the ROI formula when comparing two different trades. However, the equation does not take the time into account.
This means that in some situations, one investment may seem more profitable than the other when, in reality, its efficiency was lower because it required a much longer period. So, if Alice’s first trade had a 90% ROI but took 12 months to happen, it would be less efficient than a second trade that had, for example, a 70% ROI in 6 months.
Previous rounds:
Round 51:
Round 50:
[https://www.reddit.com/r/CryptoCurrency/comments/qz0upl/cryptotrivial_contest\
... keep reading on reddit ➡Right now I'm on the trajectory of going into a distributed systems back-end job that pays a lot of money, but my dream would really be to become a full-stack web developer freelancer one day.
Just wondering if anybody has any career advice pertaining to either two paths?
