Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with -sr option not filtering out pages specified with -fc option #1515

Closed
No-Github opened this issue Jan 2, 2024 · 1 comment · Fixed by #1720
Closed

Issue with -sr option not filtering out pages specified with -fc option #1515

No-Github opened this issue Jan 2, 2024 · 1 comment · Fixed by #1720
Assignees
Labels
Status: Completed Nothing further to be done with this issue. Awaiting to be closed. Type: Enhancement Most issues will probably ask for additions or changes.

Comments

@No-Github
Copy link

No-Github commented Jan 2, 2024

Please describe your feature request:

I encountered an issue while using the httpx tool. When I tried to save pages using the -sr option, I noticed that it was saving pages with a status code of 404. I believe that when I use the -fc option to filter out pages with 404 status codes, the -sr option should also filter out the pages specified by the -fc option.

httpx -nc -duc -l all_url.txt -silent -threads 50 -o httpx.txt -random-agent -follow-redirects -retries 3 -tlsi -fc 401,404 -sr -nfs

Describe the use case of this feature:

I made a simple code modification to demonstrate this requirement:

runner.go L1888

	if scanopts.StoreResponse || scanopts.StoreChain {
		responsePath = fileutilz.AbsPathOrDefault(filepath.Join(responseBaseDir, domainResponseFile))
		// URL.EscapedString returns that can be used as filename
		respRaw := resp.Raw
++		if len(r.options.filterStatusCode) > 0 && slice.IntSliceContains(r.options.filterStatusCode, resp.StatusCode) {
++			fmt.Println("hint fc option,pass")
++		} else {
			reqRaw := requestDump
			if len(respRaw) > scanopts.MaxResponseBodySizeToSave {
				respRaw = respRaw[:scanopts.MaxResponseBodySizeToSave]
			}
			data := reqRaw
			if scanopts.StoreChain && resp.HasChain() {
				data = append(data, append([]byte("\n"), []byte(resp.GetChain())...)...)
			}
			data = append(data, respRaw...)
			data = append(data, []byte("\n\n\n")...)
			data = append(data, []byte(fullURL)...)
			_ = fileutil.CreateFolder(responseBaseDir)
			writeErr := os.WriteFile(responsePath, data, 0644)
			if writeErr != nil {
				gologger.Error().Msgf("Could not write response at path '%s', to disk: %s", responsePath, writeErr)
			}
++		}
	}

I understand that adding this filter may lead to unexpected results for some users. Perhaps we can consider adding an option, such as '-srfc', to allow users more control over the filtering process.

@No-Github No-Github added the Type: Enhancement Most issues will probably ask for additions or changes. label Jan 2, 2024
@projectdiscovery projectdiscovery deleted a comment from acidvegas May 19, 2024
@Mzack9999 Mzack9999 linked a pull request May 23, 2024 that will close this issue
@Mzack9999 Mzack9999 self-assigned this May 23, 2024
@ehsandeep ehsandeep added the Status: Completed Nothing further to be done with this issue. Awaiting to be closed. label May 29, 2024
@ehsandeep
Copy link
Member

this is now fixed in latest release, now -sr option respect filters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Completed Nothing further to be done with this issue. Awaiting to be closed. Type: Enhancement Most issues will probably ask for additions or changes.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants