Is there a file format that supports raster and vector at the same time?
So you can use a vector image as the background and a raster image for the smaller details like textures above the vector layer.
I've been able to convert most of our old Mapinfo tables to a QGIS format, but I'm having difficulty with a few of our more complicated tables – I'm not a GIS expert, and I don't really have the knowledge to understand what I'm doing wrong (or if I can even explain it properly!). The tables we can't seem to convert all have what I think is the same issue, in that they are each comprised of a large number of raster images that I believe are opened at the same time as the table (eg. scans of historic maps, drawings or images). One of the programs we would like to use these tables in is an externally hosted, cloud-based system, where it would be difficult to link to things on our own network (and we'd therefore have to have the tables copied onto their servers, I think?).
So yeah, I'd like to delve a bit deeper into the fundamentals of computer image processing, with special attention to spatial rasters. Especially:
- how is a raster image stored exactly?
- how are coordinates attached to a raster?
- what is gdal and should I use it?
- what good are different file formats such as geotif, jp2000, .IMG, ECW etc??
- when and why do we use data types such as integer, float etc?
- what is image brightness, contrast, hue?
- how do the standard histrogram stretching thingies work such as 'percent clip' or 'standard deviations'
- what are different colour spaces such as RGD, YMCK and when should I use which?
These topics blend between processing spatial RS images and 'normal' image processing, which I'm exploring for my photography hobby. I reckoned the two should have a fair bit in common, but now I realize how much there is I don't know.
Any suggestions welcome, also to online courses or materials. Thanks.
(xpost from r/RemoteSensing due to lack of response...)
I have a DEM with values stretching from 1301 to 2404 and I'd like to make it stretch from 0 to 2404. Is there a simple way of doing this in QGIS?
Suppose I have elevation data represented as a .TIF file. Its 1000x1000 pixels. I want it to be 2000x2000 pixels, but I don't want it to be blocks of 4 pixels with the exact same value as the 1 pixel it was before. I want the new .TIF file to be smoother. Is there a built in function for this?
Hello, so I am trying to take all the raster files in a folder and sum them all together, and have been having some trouble figuring out how to do this. I am thinking I would need to use the raster package, but I am not sure how to loop through my entire folder of raster files and add them all up. I have over 500 raster files in this folder. Is there a way to easily create a sum of all of these raster files in R? I am basically looking for a raster calculator-type function for the summing. I would use the raster calculator tool in QGIS, but I do not know how to plug in my over 500 raster files there.
I would really appreciate some help with this problem, as I am really stuck on figuring out the code, and am quite new to using GIS calculations outside of QGIS. I have tried searching online for help, but could really use some basic explanation here. Thanks!
I'm looking to convert tab files to a format compatible to arcgis. I've been using the arcgis data interoperability quick import tool but it can't seem to convert raster tab files... Does anyone know a way to identify and then convert raster .tab files?
I used QGIS with GDAL 3.3 to polygonize a raster file. Added a field using the "field calculator" to calculate the area of individual polygons. Finally to calculate the sum of the total area.
While I transferred to another laptop I was using GDAL 3.0 to do the same process as above. The final sum of the area is waaay off (~1000km2).
Can someone help me why or what is happening?
Why YSK: This is especially useful when you need to save multiple versions of a file over time and can quickly reference the date from the file name instead of “date modified” or “date created”. For example, if I save a file today, I would name it “Example Text 20211203”. If I needed to save a new version in the same day, it would be “Example Text 20211203v2”.
Putting the date at the end instead of the front allows your files to be sorted alphabetically>chronologically. Putting the date at the front will sort your files chronologically>alphabetically.
Edit 2021-12-04-0041: Wow, this really blew up. Here are some common comments/questions.
Adding hyphens or underscores can improve readability (e.g., “Example Text 2021-12-03v001”)
For those asking why label the file name with the date and why not just sort by “date created” or “date modified”, if you send a file to someone and they save it, its “date created” will be as when they save it, not the file’s actual creation date.
If you’re going to have more than 9 versions, you would want to put a zero in front (e.g., v02 or even v001 if you know you’ll be creating 100+ versions) to keep versions in order.
Edit 2021-12-04-1221: I had to turn off notifications last night because they were flooding in lol. But holy shit over 21k upvotes, and thank you stranger for the gold. I’m happy to have started this discussion whether it’s obvious to some as it’s also an eye opener to those that may not have a standard formatting scheme or could improve their system. Happy formatting, everyone!
We are trying to convert a single-band raster files with values 0s & 1s to Vector. When using the QGIS's internal algorithm 'Polygonize', the result is slightly off from a Mac than the ones from Windows. We are trying to understand the reason for the mismatch but couldn't zero-in on any. It is also not the CRS, as we have ensured that to be the same for the layer in both the OS. If you have experienced something like this before or know what might be the root cause of this issue, can you please let me know?
Hi,
I'm a student and I want to develop a website to publish a raster map. On my computer, a python code will create a raster map every 15 minutes based on the current environmental conditions. I want to upload that developed map automatically to the website. The public can then view the website and check the environmental condition.
Could you help me answer the following question?
- What would be a good platform to develop a web map. I'm confused by the plethora of available options such as Leaflet, Mapbox, etc. Where should I start considering I don't have any experience with web development?
- Further, should I worry about scalability now? During the normal weather, not may will visit my academic project. But, during a natural hazard (say heavy rain), many people could visit the website. So, If needed I should be able to scale the website (or traffic handling capacity).
- If possible, I would
Thanks!
Hello! I’m currently working on my Bachelor thesis which is on InSAR ground movement studies in the Altai Mountains.
For analyzing the unwrapped LOS interferograms I would need to calibrate these. I’m doing this step manually by finding pixels with a high coherence in the study areas. The LOS-displacement on these same pixels should help me to properly calibrate the Raster. For example the value of these pixels (which didn’t move) are -10. In that case I would like to add +10 to all the values/pixels of this file. The places which didn’t move would then be around 0 and I could compare multiple LOS-Interef files. I tried doing this with the Raster Calculator and simply adding +10 to the layer but somehow the resulting layer doesn’t shift by this amount in the desired direction.
Does someone have a suggestion on how the correct expression should look?
Many thanks!
Does anyone have any resources or experience with importing ESRI Grid Rasters into ArcGIS Pro? I keep on getting errors :(.
The raster data I'm trying to import is this aridity data set from the United Nations http://www.fao.org/geonetwork/srv/en/main.home?uuid=221072ae-2090-48a1-be6f-5a88f061431a
Any help or advice would be mucho appreciated! Just started learning GIS a day ago.
Hi all, I have some rasters that I want to split the names for as I'm having issues with character numbers. For example, I have a raster called 'WRI_LU10'. I want to remove the '_LU10' part - I know how to remove file extensions using:
os.path.splitext(raster_file)[0]
but not how to split the actual file name. Any help is much appreciated!
Hello everyone, I've finished processing 69TB of data collected by Archive Team from YouTube on November/December 2021. The data encompasses metadata for 4.56B YouTube videos. The result is 4 torrent sets (totaling 2.3TB), the same data is also being uploaded to archive.org. If you need the data or wish to help seeding the magnet torrent links and technical details are bellow. Thanks to everyone already seeding the files. Some fields like category, tags, codecs and subtitles are missing as this data was not crawled by the original Archive Team crawl. Hopefully it would be captured in future crawls.
I wish you all a happy new year!
magnet:?xt=urn:btih:a8de66ae506937c0b19959a652496dff20073b57&dn=videos_minimal&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337%2fannounce&tr=http%3a%2f%2fshare.camoe.cn%3a8080%2fannounce&tr=udp%3a%2f%2ftracker.torrent.eu.org%3a451%2fannounce&tr=http%3a%2f%2ft.nyaatracker.com%3a80%2fannounce&ws=https%3a%2f%2fdl-eu.opendataapi.net%2farchiveteam-youtube-dislikes-w-metadata-2021%2f
Video flat files - 345GB
magnet:?xt=urn:btih:84e58d5bd66ba5139c94cbd8bce32fd0e70d9977&dn=videos_flat&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337%2fannounce&tr=http%3a%2f%2fshare.camoe.cn%3a8080%2fannounce&tr=udp%3a%2f%2ftracker.torrent.eu.org%3a451%2fannounce&tr=http%3a%2f%2ft.nyaatracker.com%3a80%2fannounce&ws=https%3a%2f%2fdl-eu.opendataapi.net%2farchiveteam-youtube-dislikes-w-metadata-2021%2f
Video JSON files - 1.1TB
magnet:?xt
... keep reading on reddit ➡In the past couple weeks I've had a fair few people from around the business that have had to send me various screenshots for various reasons.
So far I've had Word docs with images pasted in, JPEGs printed to PDF files, and one user that decided Excel of all things would be best for the job.
No idea how people find any of these easier than just pasting the image directly into an email/Teams message/ticket, but I'd be interested to hear some of the whacky formats people have sent you.
Not even just screenshots, just formats not fit for purpose in general :D
I have a raster layer that is elevation data of California, and would like to add an offset to all data so that all elevation data is above 0m (e.g. Death Valley has elevation ~86m below sea level). Basically I want to add like 90m to every elevation data point. What's the best way to do this? I'm currently using gdal2xyz and then will use a script or something to add the values manually, but its taking a while to execute because the file size is quite large.
I am doing a final capstone project for a GIS class and one of the requirements is 3 raster based files to work with. I am doing a fire hydrant analysis of the city I live in and was wondering what good raster data I could acquire that relates in anyway to fire/fire hydrants. I essentially need good sources to pull data from along with any ideas of what I could add relating to my subject.
I know the USGS website can pull some useful images, but I need some advice on how to search through it or manipulate the images in ENVI.
Any and all advice is welcome, I graduate after this semester!
I'm trying to decide how to best serve data layers on a map. The data will be preprocessed on the server. The workflow I'm currently trying to implement goes as follows:
Acquire satellite imagery (eg. Landsat, OHS, Worldview) for particular location
Run algorithms on bands
Reproject output and save result to disk, likely cropping to finer location
Display result on map (limiting layer to resolution of source) via custom RESTful API
Current algorithms revolve around computing indices (eg. NDVI, NBR, SI, etc) but I'll eventually be classifying pixels (eg. grass, concrete, bitchumen, etc). Each computation will be displayed on it's own map layer. The computation will maintain the resolution of the source because each pixel will hold a value, the calculated index. I imagine I will do the same when classifying what's going on at each pixel.
Initially, it seemed like a no-brainer to maintain raster throughout this workflow, yet I believe online maps are produced as vectors and every layer displayed to a user is often a vector (ie. stations, trains, shops, etc). I tried converting my rasters to GeoJSON polygons (gdal_polygonise), however the file size blew-up and it wasn't exactly a 1:1 representation (I imagine I could find a better solution?).
I'm aware online maps tile their display, aptly referred to as a 'slippy map'. Tiling seems possible with vectors and rasters so serving large GeoJSONs may not be an issue for the user?
The TLDR is when is a raster/vector best? I imagine I will be utilising both types in the longterm yet unsure what's best for processing satellite imagery. Thanks.
I merged 4 geotiff rasters that total ~100mb in size all together but the resulting merged raster ended up being 10gb. I made sure to select "rendered image" instead of "raw" as the output and didn't change any other defaults. The extent of the merged raster shouldn't be much bigger than the summed extent of the individual parts either, since it's basically 4 rectangles stacked on top of each other from north to south. The one other time I tried to merge raster files I remember having this problem as well and just ended up repeating my analysis on each of the individual rasters since they were much more manageable file sizes, but that seems inefficient.
So yeah, I'd like to delve a bit deeper into the fundamentals of computer image processing, with special attention to spatial rasters. Especially:
- how is a raster image stored exactly?
- how are coordinates attached to a raster?
- what is gdal and should I use it?
- what good are different file formats such as geotif, jp2000, .IMG, ECW etc??
- when and why do we use data types such as integer, float etc?
- what is image brightness, contrast, hue?
- how do the standard histrogram stretching thingies work such as 'percent clip' or 'standard deviations'
- what are different colour spaces such as RGD, YMCK and when should I use which?
These topics blend between processing spatial RS images and 'normal' image processing, which I'm exploring for my photography hobby. I reckoned the two should have a fair bit in common, but now I realize how much there is I don't know.
Any suggestions welcome, also to online courses or materials. Thanks.
So yeah, I'd like to delve a bit deeper into the fundamentals of computer image processing, with special attention to spatial rasters. Especially:
- how is a raster image stored exactly?
- how are coordinates attached to a raster?
- what is gdal and should I use it?
- what good are different file formats such as geotif, jp2000, .IMG, ECW etc??
- when and why do we use data types such as integer, float etc?
- what is image brightness, contrast, hue?
- how do the standard histrogram stretching thingies work such as 'percent clip' or 'standard deviations'
- what are different colour spaces such as RGD, YMCK and when should I use which?
These topics blend between processing spatial RS images and 'normal' image processing, which I'm exploring for my photography hobby. I reckoned the two should have a fair bit in common, but now I realize how much there is I don't know.
Any suggestions welcome, also to online courses or materials. Thanks.
I used QGIS with GDAL 3.3 to polygonize a raster file. Added a field using the "field calculator" to calculate the area of individual polygons. Finally to calculate the sum of the total area.
While I transferred to another laptop I was using GDAL 3.0 to do the same process as above. The final sum of the area is waaay off (~1000km2).
Can someone help me why or what is happening?
