Graph partition Puns

Hello everyone, I am new to Kafka and would like some advice on picking the number of topics and partitions for an upcoming project I am working on for educational purposes.

Main goal is to for users to log into a personalized feed page with various dataset graph estimations that are updated in real-time. Each user can subscribe to different datasets.

Currently the architecture looks a bit like this:

1. We have a producer that sends the estimators for the datasets to kafka.
2. We have a websocket service where clients connect to subscribing to the required dataset estimations.
3. The websocket service is also a kafka subscriber, it always gets the latest dataset estimation from kafka and sends it to the subscribed clients.

The clients should always be able to see the latest available estimation, and any subsequent updates to them so the service should always keep track of the latest ones to send when clients connect to it.

What I am having some trouble with is deciding on the topic/partition scheme for the estimation messages. Since we want consumers to always get the latest estimation, the messages should probably be key-ed by the dataset identifier so that we don't get out of order estimations.

Other than that I am not sure if i should make multiple topics - one for each dataset or a single topic with few or many partitions.

I am currently leading to going with a single topic since due to the key ordering restriction I would probably end up making multiple single-partition topics if going with the multiple topic scheme.

For the single topic scheme I am not sure how many partitions I should go for. Since currently the only consumer is the websocket service, throughput is limited by the number of clients the service can broadcast to.

If we want to scale this the most natural next step would be to scale the websocket service by making more instances each belonging to a separate consumer group. (perhaps behind a websocket aware load balancer)

Having multiple consumers per consumer group is probably not very useful in this case since scaling on the number of clients by adding more consumer groups would probably be more effective.

Going with multiple partitions I guess I would directly map each dataset to a partition manually, this may be an operational issue when adding more datasets in the future.

All in all I am not sure if I should go with a single or multiple partitions on the single topic, I suspect the biggest factor in the d

My apologies if this is a basic question in combinatorics, but I'm a PDE theorist who just so happens to be thinking about this problem.

Suppose I have a graph G=(V,E) with n vertices and let m divide n. I want to create a disjoint partition of V into m sets (V_1, V_2, ... , V_m) each of size n/m subject to the constraint that if v_i and v_j both in V_k, then (v_i,v_j) is in E.

I'd like to be able to count the number of possible disjoint partitions subject to this constraint. Are there results of this type known? If so, could anyone point me to a reference?

I'm looking for a graph that would allow us to choose positive natural weights for all edges so that:

(1) For some N and for each vertex v, the sum of the weights of all edges incident to v is equal to N.

(2) For each vertex v, the weights of all edges incident to v form an [integer partition](https://en.wikipedia.org/wiki/Partition_(number_theory)) of N that is not shared by another vertex.

Here is an illustration.

In the illustration N=8. For every vertex, we have the same partition 1+2+5, so requirement (2) isn't satisfied.

I know that complete graphs can't satisfy (1) and (2). I'm starting to think that no graphs can, but I don't have a proof.

What do you think?

Hi, I'm looking at the ways of partitioning the graphs. I found gpmetis online to do so. I was able to successfully partition a graph as per my requirement( edgecut =2). However I want to know the vertices where the edgecut has been made by the Metis. How do I figure it out? Should I use any options to view the vertices involved in the cut? If not Metis, any other alternatives? Thanks

Does anyone here work and keep up to date with the current research on graph partitioning methods? I'm looking to a way to partition a graph and minimize edges between partitions. The most common algorithm seems to be METIS, its implemented in a lot of different libraries like DGL, but it seems to be bounded to one machine

I'm looking to partition up to a trillion edges, so seeing if there's consensus on a distributed algorithm / framework for the graph partitioning problem. If not I'll try running METIS on an x1e.32xlarge host with 4tb of ram and see if that's enough.

The end goal is to build large knowledge graphs for building graph neural networks

I wrote this tiny (~350 lines of code) lossless compression algorithm based on partitioning a graph into clusters: https://github.com/rand3289/GraphCompress

I tried using it on data from /dev/urandom and of course the graph metadata exceeds compression... I have not tried it on other types of data yet.

The algorithm is very simple: As a file is read, it represents a path through a graph. Later I partition the graph into clusters and optimize to have the least number of hyper edges (edges between clusters). This way internal vertexes can be represented as cluster indexes instead of a global vertex ID (go from 16 bit to 8 bit). This action creates lots of bytes with 0 and the file can now be compressed with any compression utility.

I do not know much about compression and was wondering if this is an existing technique?

Graph Partitioning is usually done in such a way to minimize the difference in the edge weights.

If the adjacency matrix for the graph is A then I want to find a partitioned graph A' that minimizes the error on average between (Ax - A'x). Is that the same thing as partitioning based on minimizing the edge cuts? It seems like minimizing the edge cuts is minimizing the L1 norm of (A-A'), but minimizing the squared error is minimizing the L2 norm of (A-A'). Does spectral partitioning do both?

Let's say that I have a group of 100 people. Let's label them 1,2,3..100. People talk to some people and don't talk to others. For example 1 talks with 5,18,19,54,89 and does not talk with the others. How do I split this group of 100 people into two parts A and B of 50 people each such that cross talk from group A to group B is minimized. (Cross talk is when let's say 2 talks to 8 who belong to different groups)

I have been pondering about this problem for a long time. Please enlighten me with some concepts that might help in solving this. Comment for any clarification.

About a month ago I compiled a long crypto dictionary for people who are either new, out of the loop or just lurking around. As we are witnessing a massive influx of new crypto investors and users here, I was asked by the Redditors to post the updated version after Christmas. After somehow catching COVID a few days ago, I have a lot of free time on my hands so I decided to update the dictionary with all the suggestions people added in the previous post and a lot of new content to help out the newbies and veterans alike.

With expressions like "HODL", "FOMO", "DAO", "ATH", "LP" we can sometimes sound alien to people who aren't into cryptospace.

Are you new to crypto and want to join in the conversation? Lets go boys and girls, grab your cocoa and get cosy cause this is gonna be a long ride. ^((ps. please send some bandages for my fingers and carpal tunnel treatment braces for my poor wrists).)

Hello all,

I've got a problem and I'm trying to determine whether there is already an algorithm or equation out there that will solve this problem, or if I will have to engineer one myself.

(Note: I'm not asking for anyone to write any code or do any math for me and this isn't for a class. Just looking for the name of the algorithm, if it exists).

Here is an abstracted, general description of the problem:

-You have a grid of blocks. Each block has a given number of dots in it, and the dots are color-coded in one of two colors. Find the set of block groups such that:

1. All blocks are used
2. No block can be used more than once
3. You can only make groups of blocks that are next to each other (if block a <-> block b <-> block c, but a is not connected to c, a <-> b <-> c is still a valid combo.)
4. The number of combined dots in the block group cant be over or under a certain threshold
5. There has to be exactly a certain given number of block groups (in this case 150)
6. The final set of block groups returned is the one where, given all of the other conditions, each group has the most equal balance between the colors of dots.

(I suspect the answer lies somewhere in graph theory, I just don't know enough of it to know whether this exists or not. My background is in data science/analytics and computational journalism.)

Thanks!

I was learning about NP problems and ran into k-way graph partitioning. Basically we are trying to make sets of max size n while trying to reduce the number of edges deleted from the partitioning. How would graph partitioning work if you didn't want certain nodes together in the same partition?

Sparsest cut, and its close variant "balanced cut", have been studied for ages in the computer science theory community, and there are algorithms to compute this which run fast. Sparsest cut is often billed as a "graph clustering" algorithm.

Sparsest cut gives rise to an obvious data clustering algorithm: build a graph on your data (like kNN or threshold graph), and run sparsest cut to get a 2-way partition. To do a k-way partition, you could recursively do this. From what I know, sparsest cut intuition is one thing that guided the creation of spectral clustering, a widely used clustering method.

HHas anyone tried implementing this in practice? If so, how does it do on data? If not, why not? I have searched extensively on Google for any implementations of this sparsest-cut style clustering, but I haven't found any. Given that sparsest cut is one of the most well-studied problems in computer theory, I was surprised that no implementation of this clustering method exists.

I want to find the green cut in this picture and the function should return 2.

https://upload.wikimedia.org/wikipedia/commons/c/c0/Min_cut_example.svg

My Graph has no flows and is undirected. The Picture is from Karger's algorithm but it seems the algorithm is random and I need correct values ( I don't care about runtime)

In Networkx I found this https://networkx.github.io/documentation/networkx-1.10/reference/generated/networkx.algorithms.flow.minimum_cut.html but it seems that is a different mincut with flows and a start and target node, not for the whole graph

I've been reading about the algebraic connectivity of graphs (aka fiedler vector/eigenvector) and how people use it to partition graphs; but none of the examples or papers show how to map components of the fiedler vector to nodes in the graph (or the edges to cut). Does anyone know of a good review paper or blog post on this?

http://imgur.com/GPeLDft

"9y^2 = 4x^3 from x=0 to x=1; y> or = 0"

Not sure where I'm struggling here... I know I got the formula part right but I seem to be doing something wrong in my calculation and/or finding the antiderivative. Correct answer is (4sqrt(2) - 2)/3

I don't want to step on anybody's toes here, but the amount of non-dad jokes here in this subreddit really annoys me. First of all, dad jokes CAN be NSFW, it clearly says so in the sub rules. Secondly, it doesn't automatically make it a dad joke if it's from a conversation between you and your child. Most importantly, the jokes that your CHILDREN tell YOU are not dad jokes. The point of a dad joke is that it's so cheesy only a dad who's trying to be funny would make such a joke. That's it. They are stupid plays on words, lame puns and so on. There has to be a clever pun or wordplay for it to be considered a dad joke.

Again, to all the fellow dads, I apologise if I'm sounding too harsh. But I just needed to get it off my chest.

I've been fiddling with 2D games for awhile and I'm trying to go into 3D game development. I thought I should get my basics right first.

From what I read scene graphs hold your game objects/entities and their relation to each other like 'a tire' would be the child of 'a vehicle'. It's mainly used for frustum/occlusion culling and minimizing the collision checks between the objects.

Spatial partitioning structures on the other hand are used to divide a big game object (like the map) to smaller parts so that you can gain performance by only drawing the relevant polygons and again minimizing the collision checks to those polygons only. Also a spatial partitioning data structure can be used as a node in a scene graph.

But... I've been reading about both subjects and I've seen a lot of "scene graphs are useless" and "BSP performance gain is irrelevant with modern hardware" kind of articles.

Also some of the game engines I've checked like gameplay3d and jmonkeyengine are only using a scene graph (That also may be because they don't want to limit the developers). Whereas games like Quake and Half-Life only use spatial partitioning.

I'm aware that the usage of these structures very much depend on the type of the game you're developing so for the sake of clarity let's assume the game is a FPS like Counter-Strike with some better outdoor environment capabilities (like a terrain).

The obvious question is which one is needed and why (considering the modern hardware capabilities).

Thank you.

So I've been playing around on vanspace... like way too much time haha (especially since I'm not digitally savvy in that way and prefer my graph paper and pencils) anyway- I have a few different layouts for my new build and the one I am currently leaning towards is not one I've seen in other vans. I think I have a screenshot of one sort of like it in the UK but forgot to include where I found it.

I am doing a 148 high roof transit- but NOT extended, which means it is a much smaller space than so many layouts these days show. It is plenty room for me and a huge upgrade from my current regular roof van ford e350.

In all of the imaginings of what I need to be able to do in the van (including work, with a zoom background that is NOT my bed) and the other logistics, I really want to have seats with a window view and also when door is open, but I don't want to use the layout with the seats on the driver side as I feel it takes up too much space.

I am going to do a partition with an opening/ door, no swivel seats. And yes I know if I did swivel seats it would take care of this problem and give me more room, but I have my reasons.

So just wanted to see if anyone has done something like this, and do you have photos? Have you tried it and it didn't work or you did and love it?

Obviously I would make use of the box bench seats and figure out how to correctly situation either a Lagun table or something. I'm also thinking about designing a swivel back on the box at the door front so it have the back move to face outside or in...

OK what do you think? (the kitchen would be on the opposite wall) (also ignore the shelves and plants I was just playing around for vibe that's not technical at all haha)

View of the passenger side sliding door (outline of door should be around that window)

Hi guys, I came across an interesting problem today and I'd like some help solving it.

>Given a graph, partition its vertices into two non-empty >sets of equal sizes and make sure that the crossing edges have >minimum weight.

I have been reading about min-cut algorithms, especially this but I thought that asking for help or more resources here would be beneficial.

Any help regarding how I solve this would be appreciated. Thanks.

Do your worst!

They were cooked in Greece.

I'm surprised it hasn't decade.

For context I'm a Refuse Driver (Garbage man) & today I was on food waste. After I'd tipped I was checking the wagon for any defects when I spotted a lone pea balanced on the lifts.

I said "hey look, an escaPEA"

No one near me but it didn't half make me laugh for a good hour or so!

Edit: I can't believe how much this has blown up. Thank you everyone I've had a blast reading through the replies 😂